System and method for adaptive intra refresh rate setting in a video encoder

Abstract

A system and method for adaptive intra refresh rate setting for each video frame in a video encoder is disclosed. In one embodiment, a video encoding method includes computing channel induced distortion values at multiple intra refresh rates for a coded video frame as a function of channel loss and spatiotemporal content, computing source coding distortion values at the multiple intra refresh rates for the coded video frame as a function of the spatiotemporal content and an operating bit-rate, selecting an intra refresh rate value for one or more video frames subsequent to the encoded video frame as a function of the sum of the corresponding computed channel induced distortion values and the source coding distortion values, and encoding the one or more video frames using the selected intra refresh rate value.

What is claimed is: 
     
       1. A video encoding method, comprising:
 computing channel induced distortion values at multiple intra refresh rates for a coded video frame as a function of channel loss and spatiotemporal content; 
 computing source coding distortion values at the multiple intra refresh rates for the coded video frame as a function of the spatiotemporal content and an operating bit-rate; 
 selecting an intra refresh rate value for one or more video frames subsequent to the encoded video frame as a function of the sum of the corresponding computed channel induced distortion values and the source coding distortion values; and 
 encoding the one or more video frames using the selected intra refresh rate value. 
 
     
     
       2. The method of  claim 1 , wherein computing the channel induced distortion values at multiple intra refresh rates for the coded video frame as a function of channel loss and spatiotemporal content, comprises;
 modeling channel induced distortion as a function of the channel loss, spatiotemporal content, and an intra refresh rate; and 
 computing the channel induced distortion values at multiple intra refresh rates using the modeled channel induced distortion for the coded video frame. 
 
     
     
       3. The method of  claim 2 , wherein the channel induced distortion is modeled using the equation:
     A (β)= B−C (β),
 
 wherein A represents estimated channel induced distortion in a frame when coded with an intra-refreshing rate of β refresh macroblocks (MBs)/frame, B represents estimated channel induced distortion in the frame n when no MBs are refreshed, and C represents reduction in the channel induced distortion with β refresh MBs. 
 
     
     
       4. The method of  claim 2 , wherein the channel induced distortion is modeled using the equation: 
       
         
           
             
               
                 
                   D 
                   ep_est 
                   Fr 
                 
                 ⁡ 
                 
                   ( 
                   
                     n 
                     , 
                     β 
                   
                   ) 
                 
               
               = 
               
                 
                   
                     D 
                     ep_est 
                     Fr 
                   
                   ⁡ 
                   
                     ( 
                     
                       n 
                       , 
                       0 
                     
                     ) 
                   
                 
                 - 
                 
                   
                     ( 
                     
                       1 
                       - 
                       p 
                     
                     ) 
                   
                   ⁢ 
                   
                     
                       ∑ 
                       
                         i 
                         = 
                         1 
                       
                       
                         i 
                         = 
                         β 
                       
                     
                     ⁢ 
                     
                       
                         D 
                         ep_sorted 
                         MB 
                       
                       ⁡ 
                       
                         ( 
                         i 
                         ) 
                       
                     
                   
                 
               
             
           
         
         where, D ep     —     est   Fr (n,β) is the estimated channel induced distortion in a frame n when coded with the intra-refreshing rate of β refresh MBs/frame, D ep     —     sorted   MB (i) is the i th  value in a list of MB-level channel induced distortion estimates when sorted in descending order of their values, where the summation extends over i=1 to β, D ep     —     est   Fr (n,0) is the estimated channel induced distortion in the frame n when no MBs are refreshed, and p is the packet loss probability in the channel. 
       
     
     
       5. The method of  claim 4 , wherein the channel induced distortion in the frame n when no MBs are refreshed, is computed using the equation:
     D   ep     —     est   Fr ( n, 0)=(1− p ) bD   ep   Fr ( n− 1)+ p[D   ep   Fr ( n− 1)+ D   ec   Fr ( n )]
 
 where, D ec   Fr (n) is the distortion occurring because of error concealment in the frame n, D ep   Fr (n−1) is the estimated channel induced distortion in the frame n−1, and b is a motion randomness factor which indicates the extent of information taken from the previous frame due to motion compensation and is computed using the equation:
     b=ΣD   ep   mc ( n− 1, m )/ D   ep   fr ( n− 2) 
 
 where, D ep   Fr (n−2) is the estimated channel induced distortion in the frame n−2, and ΣD ep   mc (n−1,m) is the motion compensated error propagation distortion summed over the pixels in the frame (n−2) to which the m th  MB in the frame (n−1) refers to for motion compensated prediction (MCP) and the summation extends over all inter coded MBs in the frame (n−1). 
 
     
     
       6. The method of  claim 4 , wherein encoding the one or more video frames using the selected intra refresh rate value, comprises:
 identifying a MB level channel induced distortion threshold value corresponding to the selected intra fresh rate value in the sorted MB-level channel induced distortion estimates; and 
 encoding a MB as intra if the MB's channel induced distortion estimate exceeds the MB level channel induced distortion threshold value. 
 
     
     
       7. The method of  claim 1 , wherein computing the source coding distortion values at multiple intra refresh rates for the coded video frame as a function of spatiotemporal content and operating bit-rate, comprises:
 modeling source coding distortion as a function of the spatiotemporal content, operating bit-rate, and an intra refresh rate; and 
 computing the source coding distortion values at multiple intra refresh rates using the modeled source coding distortion for the coded video frame. 
 
     
     
       8. The method of  claim 7 , wherein the source coding distortion is modeled using the equation:
     D   q (β)=σ 2   e   −α(R−(μ     IR     −η)β/θ)  
 
 where, D q (β) is the source-coding distortion, σ 2  is the variance of the inter-coded residuals of the frame, α and θ are model parameters and are computed using data from the previous frames, R is the total bits allocated for coding the residuals in the frame, μ IR  is an average number of bits required for coding an intra MB computed over a past window of frames, and η is an average number of bits required for coding a refresh MB as INTER-MB. 
 
     
     
       9. The method of  claim 1 , wherein selecting the intra refresh rate value comprises:
 summing the corresponding source coding and channel induced distortion values over a range of intra refresh rates from 0 to a number of macroblocks in the subsequent video frame; and 
 selecting the intra refresh rate value as a function of the summed corresponding source coding and channel induced distortion values. 
 
     
     
       10. The method of  claim 1 , wherein selecting the intra refresh rate value comprises:
 summing the corresponding source coding and channel induced distortion values over a range of intra refresh rates substantially surrounding the previously computed refresh rate; and 
 selecting the intra refresh rate value as a function of the summed corresponding source coding and channel induced distortion values. 
 
     
     
       11. The method of  claim 1 , wherein the spatiotemporal content comprises attributes selected from the group consisting of texture, motion and color. 
     
     
       12. The method of  claim 1 , wherein the channel loss comprises probability of packet loss during video transmission over a predetermined interval. 
     
     
       13. The method of  claim 1 , wherein computing the channel induced distortion values at multiple refresh rates for the coded video frame as a function of channel loss and spatiotemporal content, comprises:
 computing a channel induced distortion value as an average of the channel induced distortion values estimated over a predetermined window of video frames subsequent to the current video frame. 
 
     
     
       14. The method of  claim 1 , wherein computing the source coding distortion values at multiple refresh rates for the coded video frame as a function of spatiotemporal content and operating bit-rate comprises:
 computing an effective intra refresh rate value as a function of selected intra refresh rate values associated with multiple encoded video frames over a predetermined interval; and 
 computing the source coding distortion values as a function of the computed effective intra refresh rate value. 
 
     
     
       15. The method of  claim 1 , wherein computing the source coding distortion values at multiple refresh rates for the coded video frame as a function of bit-rate and spatiotemporal content, comprises:
 computing an effective intra refresh rate value for each of the frames in a predetermined window of video frames subsequent to the current video frame as a function of selected intra refresh rate values associated with multiple preceding video frames over a predetermined interval; and 
 computing a source coding distortion value as an average of the source coding distortion values estimated for each of the frames in the predetermined window of video frames at their corresponding effective intra refresh rate values. 
 
     
     
       16. An article comprising a non-transitory computer readable storage medium having instructions thereon which when executed by a computing platform result in execution of a video encoding method, comprising:
 computing channel induced distortion values at multiple intra refresh rates for a coded video frame as a function of channel loss and spatiotemporal content; 
 computing source coding distortion values at the multiple intra refresh rates for the coded video frame as a function of the spatiotemporal content and an operating bit-rate; 
 selecting an intra refresh rate value for one or more video frames subsequent to the encoded video frame as a function of the sum of the corresponding computed channel induced distortion values and the source coding distortion values; and 
 encoding the one or more video frames using the selected intra refresh rate value. 
 
     
     
       17. The article of  claim 16 , wherein computing the channel induced distortion values at multiple intra refresh rates for the coded video frame as a function of channel loss and spatiotemporal content, comprises;
 modeling channel induced distortion as a function of the channel loss, spatiotemporal content, and an intra refresh rate; and 
 computing the channel induced distortion values at multiple intra refresh rates using the modeled channel induced distortion for the coded video frame. 
 
     
     
       18. The article of  claim 16 , wherein computing the source coding distortion values at intra multiple refresh rates for the coded video frame as a function of spatiotemporal content and operating bit-rate, comprises:
 modeling source coding distortion as a function of the spatiotemporal content, operating bit-rate, and an intra refresh rate; and 
 computing the source coding distortion values at multiple intra refresh rates using the modeled source coding distortion for the coded video frame. 
 
     
     
       19. The article of  claim 16 , wherein computing the channel induced distortion values at multiple refresh rates for a coded video frame as a function of channel loss and spatiotemporal content, comprises:
 computing channel induced distortion value as an average of the channel induced distortion values estimated over a predetermined window of video frames subsequent to the current video frame. 
 
     
     
       20. The article of  claim 16 , wherein computing the source coding distortion values at multiple refresh rates for the coded video frame as a function of spatiotemporal content and operating bit-rate comprises:
 computing an effective intra refresh rate value as a function of selected intra refresh values associated with multiple encoded video frames over a predetermined interval; and 
 computing the source coding distortion values as a function of the computed effective intra refresh rate value. 
 
     
     
       21. The article of  claim 16 , wherein computing the source coding distortion values at multiple refresh rates for the coded video frame as a function of bit-rate and spatiotemporal content, comprises:
 computing an effective intra refresh rate value for each of the frame in a predetermined window of video frames subsequent to a current video frame as a function of selected intra refresh values associated with multiple preceding video frames over a predetermined interval; and 
 computing a source coding distortion value as an average of the source coding distortion values estimated for each of the frames in the predetermined window of video frames at their corresponding effective intra refresh rate values. 
 
     
     
       22. A video encoder, comprising:
 a video encoding module; and 
 a refresh rate selection module coupled to the video encoding module, wherein the refresh rate selection module comprises: 
 a source coding distortion module to receive bit consumption information at an operating bit-rate and spatiotemporal content attributes from the video encoding module and compute source coding distortion values at multiple intra refresh rates for a coded video frame as a function of the received spatiotemporal content attributes and bit consumption information at the operating bit-rate; 
 a channel induced distortion module to receive channel loss information from one or more receivers and video frame level error propagation estimates from the video encoding module and compute channel induced distortion values at multiple intra refresh rates for the coded video frame as a function of the received channel loss information and spatiotemporal content having the video frame level error propagation estimates; and 
 an intra refresh rate selection unit to select an intra refresh rate value for one or more video frames subsequent to the encoded video frame as a function of the sum of the corresponding computed channel induced distortion values and the source coding distortion values, wherein the video encoding module encodes the one or more video frames using the selected intra refresh rate value. 
 
     
     
       23. The video encoder of  claim 22 , wherein the source coding distortion module to model source coding distortion as a function of the spatiotemporal content attributes, the bit consumption information at the operating bit-rate, and an intra refresh rate, and compute the source coding distortion values at multiple intra refresh rates using the modeled source coding distortion for the coded video frame. 
     
     
       24. The video encoder of  claim 22 , wherein the channel induced distortion module to model channel induced distortion as a function of the channel loss information, spatiotemporal content having the video frame level error propagation estimates, and an intra refresh rate, and compute the channel induced distortion values at multiple intra refresh rates using the modeled channel induced distortion for the coded video frame.