Frame interpolation using more accurate motion information

Abstract

In general, this disclosure describes encoding and decoding techniques that facilitate more accurate interpolation of skipped video frames. In particular, techniques are described for obtaining motion information that indicates motion for skipped video frames based on translational motion and at least one other motion parameter and applying the motion information to interpolate the skipped video frames. The motion information may, for example, indicate motion based on a motion model that models three or more motion parameters as opposed to conventional two parameter translational motion vectors. The more accurate motion information may either be generated within the decoder performing the interpolation or be transmitted by an encoder in one or more frames. Either way, the techniques reduce the amount of visual artifacts in the interpolated frame.

1 . A method for processing digital video data, the method comprising: 
 obtaining motion information that indicates motion for a skipped video frame based on translational motion and at least one other motion parameter; and    applying the motion information to interpolate the skipped video frame.    
   
   
       2 . The method of  claim 1 , wherein the motion information indicates motion based on an affine motion model.  
   
   
       3 . The method of  claim 1 , wherein obtaining the motion information comprises receiving a plurality of digital video frames, wherein the motion information is encoded within at least one of the received video frames.  
   
   
       4 . The method of  claim 1 , wherein obtaining the motion information comprises: 
 receiving motion information associated with one or more video frames adjacent to the skipped video frame; and    generating the motion information for the skipped video frame based on motion information associated with the video frames adjacent the to skipped video frame.    
   
   
       5 . The method of  claim 1 , wherein the motion information comprises first motion information, the method further comprising receiving second motion information associated with one or more video frames adjacent to the skipped video frame, wherein applying the motion information comprises applying the first and second motion information to interpolate the skipped video frame.  
   
   
       6 . The method of  claim 1 , wherein the motion information indicates motion for the entire skipped video frame.  
   
   
       7 . The method of  claim 1 , wherein the motion information indicates motion for a portion of the skipped video frame, the method further comprising: 
 receiving location information that describes the portion of the skipped video frame associated with the motion information; and    applying the motion information to interpolate the portion of the skipped video frame described by the location information.    
   
   
       8 . The method of  claim 1 , further comprising converting the received motion information to motion information that indicates motion for the skipped video frame based only on translational motion.  
   
   
       9 . The method of  claim 8 , wherein converting the received motion information to motion information that indicates motion for the skipped video frame based only on translational motion comprises: 
 generating motion vectors based on the motion information for one or more pixels within a block of pixels of interest; and    merging the motion vectors of the one or more pixels to produce a motion vector for the entire block of pixels.    
   
   
       10 . The method of  claim 1 , wherein the other motion parameter comprises at least one of scaling, shearing, rotation, panning and tilting.  
   
   
       11 . A processor for processing digital video data, the processor being configured to obtain motion information that indicates motion for a skipped video frame based on translational motion and at least one other motion parameter, and apply the motion information to interpolate the skipped video frame.  
   
   
       12 . The processor of  claim 11 , wherein the processor is configured to obtain motion information that indicates motion based on an affine motion model.  
   
   
       13 . The processor of  claim 11 , wherein the processor is configured to receive the motion information encoded within at least one received video frame.  
   
   
       14 . The processor of  claim 11 , wherein the processor is configured to: 
 receive motion information associated with one or more video frames adjacent to the skipped video frame; and    generate the motion information for the skipped video frame based on motion information associated with the video frames adjacent to the skipped video frame.    
   
   
       15 . The processor of  claim 11 , wherein the motion information comprises first motion information, and the processor is further configured to receive second motion information associated with one or more video frames adjacent to the skipped video frame and apply the first and second motion information to interpolate the skipped video frame.  
   
   
       16 . The processor of  claim 11 , wherein the processor is configured to receive motion information that indicates motion for a portion of the skipped video frame, receive location information that describes the portion of the skipped video frame associated with the motion information, and apply the motion information to interpolate the portion of the skipped video frame described by the location information.  
   
   
       17 . The processor of  claim 11 , wherein the processor is configured to convert the received motion information to motion information that indicates motion for the skipped video frame based only on translational motion.  
   
   
       18 . The processor of  claim 11 , wherein the processor is incorporated within a wireless communication device, the device further comprising a receiver to receive digital video frames at least one of which is used by the processor to interpolate the skipped video frame.  
   
   
       19 . An apparatus for processing digital video data, the apparatus comprising an interpolation module that obtains motion information that indicates motion for a skipped video frame based on translational motion and at least one other motion parameter, and applies the motion information to interpolate the skipped video frame.  
   
   
       20 . The apparatus of  claim 19 , wherein the interpolation module obtains motion information that indicates motion based on an affine motion model.  
   
   
       21 . The apparatus of  claim 19 , wherein the interpolation module receives the motion information encoded within at least one received video frame.  
   
   
       22 . The apparatus of  claim 19 , further comprising a motion estimation module that receives motion information associated with one or more video frames adjacent to the skipped video frame and generates the motion information for the skipped video frame based on motion information associated with the video frames adjacent the to skipped video frame, wherein the interpolation module obtains the motion information from the motion estimation module.  
   
   
       23 . The apparatus of  claim 19 , further comprising a motion information conversion module that converts the motion information to motion information that indicates motion for the skipped video frame based only on translational motion.  
   
   
       24 . An apparatus for processing digital video data, the apparatus comprising: 
 means for obtaining motion information that indicates motion for a skipped video frame based on translational motion and at least one other motion parameter; and    means for interpolating the skipped video frame by applying the motion information.    
   
   
       25 . The apparatus of  claim 24 , wherein the motion information indicates motion based on an affine motion model.  
   
   
       26 . The apparatus of  claim 24 , further comprising means for receiving a plurality of digital video frames, wherein the motion information is encoded within at least one of the received video frames.  
   
   
       27 . The apparatus of  claim 24 , further comprising: 
 means for receiving motion information associated with one or more video frames adjacent to the skipped video frame; and    means for generating the motion information for the skipped video frame based on motion information associated with the video frames adjacent to the skipped video frame.    
   
   
       28 . The apparatus of  claim 24 , wherein the motion information indicates motion for a portion of the skipped video frame, and further comprising means for receiving location information that describes the portion of the skipped video frame associated with the motion information, and further wherein the interpolating means applies the motion information to interpolate the portion of the skipped video frame described by the location information.  
   
   
       29 . The apparatus of  claim 24 , further comprising means for converting the received motion information to motion information that indicates motion for the skipped video frame based only on translational motion.  
   
   
       30 . A machine-readable medium comprising instructions that upon execution cause a machine to: 
 obtain motion information that indicates motion for a skipped video frame based on translational motion and at least one other motion parameter; and    apply the motion information to interpolate the skipped video frame.    
   
   
       31 . The machine-readable medium of  claim 30 , wherein instructions that cause the machine to obtain the motion information comprises instructions to receive a plurality of digital video frames, wherein the motion information is encoded within at least one of the received video frames.  
   
   
       32 . The machine-readable medium of  claim 30 , wherein instructions that cause the machine to obtain the motion information comprises instructions to: 
 receive motion information associated with one or more video frames adjacent to the skipped video frame; and    generate the motion information for the skipped video frame based on motion information associated with the video frames adjacent to the skipped video frame.    
   
   
       33 . The machine-readable medium of  claim 30 , further comprising instructions to convert the received motion information to motion information that indicates motion for the skipped video frame based only on translational motion.  
   
   
       34 . A video encoding method comprising: 
 generating motion information that indicates motion for a skipped video frame based on translational motion and at least one other motion parameter; and    encoding the motion information within at least one video frame.    
   
   
       35 . The method of  claim 34 , wherein generating motion information comprises generating motion information that indicates motion based on an affine motion model.  
   
   
       36 . The method of  claim 34 , wherein generating motion information comprises generating motion information that indicates motion for a portion of the skipped video frame, the method further comprising: 
 generating location information that describes the portion of the skipped video frame associated with the motion information; and    encoding the location information within the video frame.    
   
   
       37 . The method of  claim 36 , wherein generating location information comprises: 
 performing motion segmentation to identify objects within the skipped frame with motion other than translational motion; and    generating the motion information based on translational motion and at least one other motion parameter for the identified objects.    
   
   
       38 . The method of  claim 34 , wherein generating motion information that indicates motion for the skipped video frame based on translational motion and at least one other motion parameter comprises generating motion information that indicates motion for a skipped video frame based on translational motion and at least one of scaling, shearing, rotation, panning and tilting.  
   
   
       39 . The method of  claim 34 , wherein encoding the motion information within at least one video frame comprises encoding the motion information within a non-skipped video frame.  
   
   
       40 . The method of  claim 34 , further comprising transmitting the video frame to a video decoder to assist the video decoder in interpolation of the skipped video frame.  
   
   
       41 . An apparatus for encoding digital video data, the apparatus comprising: 
 an analysis module that analyzes a skipped video frame and generates motion information that indicates motion for the skipped video frame based on translational motion and at least one other motion parameter; and    an assembly module that encodes the motion information within at least one video frame.    
   
   
       42 . The apparatus of  claim 41 , wherein the analysis module generates motion information that indicates motion based on an affine motion model.  
   
   
       43 . The apparatus of  claim 41 , wherein the analysis module generates motion information that indicates motion for a portion of the skipped video frame, generates location information that describes the portion of the skipped video frame associated with the motion information, and encodes the location information within the video frame.  
   
   
       44 . The apparatus of  claim 41 , further comprising a transmitter to transmit the video frame to a video decoder to assist the video decoder in interpolation of the skipped video frame.  
   
   
       45 . The apparatus of  claim 41 , wherein the analysis module generates motion information that indicates motion for a skipped video frame based on translational motion and at least one of scaling, shearing, rotation, panning and tilting.  
   
   
       46 . An apparatus for encoding digital video data, the apparatus comprising: 
 means for generating motion information that indicates motion for a skipped video frame based on translational motion and at least one other motion parameter; and    means for assembling frames that encodes the motion information within at least one video frame.    
   
   
       47 . The apparatus of  claim 46 , wherein the generation means generates motion information that indicates motion for a portion of the skipped video frame, generates location information that describes the portion of the skipped video frame associated with the motion information, and encodes the location information within the video frame.  
   
   
       48 . The apparatus of  claim 47 , wherein the generation means generates motion information based on translational motion, performs motion segmentation to identify objects within the skipped frame with motion other than translational motion, and generates the motion information based on translational motion and at least one other motion parameter for the identified objects.  
   
   
       49 . The apparatus of  claim 46 , wherein the generation means generates motion information that indicates motion for a skipped video frame based on translational motion and at least one of scaling, shearing, rotation, panning and tilting.  
   
   
       50 . A processor for encoding digital video data, the processor being configured to: 
 generate motion information that indicates motion for a skipped video frame based on translational motion and at least one other motion parameter; and    encode the motion information within at least one video frame.    
   
   
       51 . The processor of  claim 50 , wherein the processor is configured to generate motion information comprises generating motion information that indicates motion based on an affine motion model.  
   
   
       52 . The processor of  claim 50 , wherein the processor is configured to: 
 generate location information that describes the portion of the skipped video frame associated with the motion information; and    encode the location information within the video frame.    
   
   
       53 . The processor of  claim 50 , wherein the processor is configured to encode the motion information within a non-skipped video frame.  
   
   
       54 . A machine-readable medium comprising instructions that upon execution cause a machine to: 
 generate motion information that indicates motion for a skipped video frame based on translational motion and at least one other motion parameter; and    encode the motion information within at least one video frame.    
   
   
       55 . The machine-readable medium of  claim 54 , wherein the motion information indicates motion based on an affine motion model.  
   
   
       56 . The machine-readable medium of  claim 54 , wherein instructions that cause the machine to generate motion information comprise instructions to generate motion information that indicates motion for a portion of the skipped video frame, further comprising instructions to: 
 generate location information that describes the portion of the skipped video frame associated with the motion information; and    encode the location information within the video frame.    
   
   
       57 . The machine-readable medium of  claim 54 , further comprising instructions that cause the machine to encode the motion information within a non-skipped video frame.