Nonlinear, prediction filter for hybrid video compression

Abstract

A method and apparatus for non-linear prediction filtering are disclosed. In one embodiment, the method comprises performing motion compensation to generate a motion compensated prediction using a block from a previously coded frame, performing non-linear filtering on the motion compensated prediction in the transform domain with a non-linear filter as part of a fractional interpolation process to generate a motion compensated non-linear prediction, subtracting the motion compensated non-linear prediction from a block in a current frame to produce a residual frame, and coding the residual frame.

1 . A method comprising: 
 performing motion compensation to generate a motion compensated prediction using a block from a previously coded frame;    performing non-linear filtering on the motion compensated prediction in the transform domain with a non-linear filter as part of a fractional interpolation process to generate a motion compensated non-linear prediction;    subtracting the motion compensated non-linear prediction from a block in a current frame to produce a residual frame; and    coding the residual frame.    
   
   
       2 . The method defined in  claim 1  further comprising applying non-linear filtering to a previously decoded frame, and wherein performing motion compensation comprises generating the motion compensated prediction using the non-linearly filtered previously decoded frame.  
   
   
       3 . The method defined in  claim 1  wherein performing non-linear filtering comprises: 
 applying a transform to the motion compensated prediction to obtain a first set of transform coefficients;    thresholding each transform coefficient based on a threshold to identify a set of one or more transform coefficients to change;    modifying the first set of transform coefficients to create a second set of transform coefficients by changing transform coefficients in the set of one or more transform coefficients; and    applying an inverse transform to the second set of transform coefficients to generate the motion compensated non-linear prediction.    
   
   
       4 . The method defined in  claim 1  wherein performing non-linear filtering comprises: 
 applying a transform to the motion compensated prediction to obtain a first set of transform coefficients;    ranking transform coefficients based on magnitude to identify a set of one or more transform coefficients to change;    modifying the first set of transform coefficients to create a second set of transform coefficients by changing transform coefficients in the set of one or more transform coefficients; and    applying an inverse transform to the second set of transform coefficients to generate the motion compensated non-linear prediction.    
   
   
       5 . The method defined in  claim 1  wherein performing non-linear filtering comprises: 
 applying a transform to the motion compensated prediction to obtain a first set of transform coefficients;    percentage filtering each transform coefficient of the block to identify a set of one or more transform coefficients to change;    modifying the first set of transform coefficients to create a second set of transform coefficients by changing transform coefficients in the set of one or more transform coefficients; and    applying an inverse transform to the second set of transform coefficients to generate the motion compensated non-linear prediction.    
   
   
       6 . The method defined in  claim 1  wherein performing non-linear filtering comprises: 
 applying a transform to the motion compensated prediction to obtain a first set of transform coefficients;    assigning an index to each transform coefficient in the block;    identifying a set of one or more transform coefficients to change based on the indices;    modifying the first set of transform coefficients to create a second set of transform coefficients by changing transform coefficients in the set of one or more transform coefficients; and    applying an inverse transform to the second set of transform coefficients to generate the motion compensated non-linear prediction.    
   
   
       7 . The method defined in  claim 1  wherein performing non-linear filtering comprises: 
 applying a transform to the motion compensated prediction to obtain a first set of transform coefficients;    scaling one or more transform coefficients to create a second set of transform coefficients; and    applying an inverse transform to the second set of transform coefficients to generate the motion compensated non-linear prediction.    
   
   
       8 . The method defined in  claim 7  wherein the scaling is performed by scaling with zero.  
   
   
       9 . The method defined in  claim 1  wherein performing non-linear filtering comprises: 
 applying a transform to the motion compensated prediction to obtain a first set of transform coefficients;    zeroing one or more transform coefficients to create a second set of transform coefficients; and    applying an inverse transform to the second set of transform coefficients to generate the motion compensated non-linear prediction.    
   
   
       10 . The method defined in  claim 1  wherein performing non-linear filtering comprises: 
 applying a transform to the motion compensated prediction to obtain a first set of transform coefficients;    performing a soft-thresholding operation on one or more transform coefficients to create a second set of transform coefficients; and    applying an inverse transform to the second set of transform coefficients to generate the motion compensated non-linear prediction.    
   
   
       11 . The method defined in  claim 10  wherein performing a soft-thresholding operation includes reordering a coefficient if its magnitude is greater than a threshold and setting the coefficient to zero if its magnitude is less than the threshold.  
   
   
       12 . The method defined in  claim 1  wherein performing non-linear filtering comprises: 
 applying a transform to the motion compensated prediction to obtain a first set of transform coefficients;    modifying coefficients by obtaining a one-dimensional increasing function of the coefficients to create a second set of transform coefficients; and    applying an inverse transform to the second set of transform coefficients to generate the motion compensated non-linear prediction.    
   
   
       13 . The method defined in  claim 1  wherein performing non-linear filtering comprises: 
 applying a transform to the motion compensated prediction to obtain a first set of transform coefficients;    applying denoising to one or more transform coefficients to create a second set of transform coefficients; and    applying an inverse transform to the second set of transform coefficients to generate the motion compensated non-linear prediction.    
   
   
       14 . The method defined in  claim 1  adding into the coded bitstream information indicative of the non-linear filter used to filter the motion compensated non-linear prediction.  
   
   
       15 . The method defined in  claim 14  further comprising adding information indicative of the transform to signal the decoder as to which transform to apply to create the transform coefficients during decoding.  
   
   
       16 . The method defined in  claim 1  wherein the non-linear filter is one of a plurality of filters, and further comprising selecting the non-linear filter from among the plurality of filters.  
   
   
       17 . The method defined in  claim 1  further comprising: 
 generating a plurality of motion compensated non-linear predictions for the video frame; and    eliminating one or more of the plurality of motion compensated non-linear predictions based on differences with previously found predictions.    
   
   
       18 . The method of  claim 1  wherein transform coefficients exceed in number the candidate block elements.  
   
   
       19 . An encoder comprising: 
 a motion compensation prediction module to perform motion compensation to generate a motion compensated prediction using a block from a previously coded frame;    a non-linear prediction filter to perform non-linear filtering on the block in the transform domain with a non-linear filter as part of a fractional interpolation process;    a subtractor to subtract the motion compensated non-linear prediction from a block in a current frame to produce a residual frame; and    a coder to code the residual frame.    
   
   
       20 . The encoder defined in  claim 19  wherein the non-linear prediction filter comprises: 
 a forward transform module to apply a transform to the motion compensated prediction to obtain a first set of transform coefficients;    a module to perform thresholding on each transform coefficient based on a threshold to identify a set of one or more transform coefficients to change and to modify the first set of transform coefficients to create a second set of transform coefficients by changing transform coefficients in the set of one or more transform coefficients; and    an inverse transform module to apply an inverse transform to the second set of transform coefficients to generate the motion compensated non-linear prediction.    
   
   
       21 . The encoder defined in  claim 19  wherein the non-linear prediction filter comprises: 
 a forward transform module to apply a transform to the motion compensated prediction to obtain a first set of transform coefficients;    a module to sort transform coefficients based on magnitude to identify a set of one or more transform coefficients to change and modify the first set of transform coefficients to create a second set of transform coefficients by changing transform coefficients in the set of one or more transform coefficients; and    an inverse transform module to apply an inverse transform to the second set of transform coefficients to generate the motion compensated non-linear prediction.    
   
   
       22 . The encoder defined in  claim 19  wherein the non-linear prediction filter comprises: 
 a forward transform module to apply a transform to the motion compensated prediction to obtain a first set of transform coefficients;    a module to perform percentage filtering each transform coefficient of the block to identify a set of one or more transform coefficients to change and to modify the first set of transform coefficients to create a second set of transform coefficients by changing transform coefficients in the set of one or more transform coefficients; and    an inverse transform module to apply an inverse transform to the second set of transform coefficients to generate the motion compensated non-linear prediction.    
   
   
       23 . The encoder defined in  claim 19  wherein the non-linear prediction filter comprises: 
 a forward transform module to apply a transform to the motion compensated prediction to obtain a first set of transform coefficients;    a module to assign an index to each transform coefficient in the block, identify a set of one or more transform coefficients to change based on the indices, and modify the first set of transform coefficients to create a second set of transform coefficients by changing transform coefficients in the set of one or more transform coefficients; and    an inverse transform module to apply an inverse transform to the second set of transform coefficients to generate the motion compensated non-linear prediction.    
   
   
       24 . The encoder defined in  claim 19  wherein the non-linear prediction filter comprises: 
 a forward transform module to apply a transform to the motion compensated prediction to obtain a first set of transform coefficients;    a module to scale one or more transform coefficients to create a second set of transform coefficients; and    an inverse transform module to apply an inverse transform to the second set of transform coefficients to generate the motion compensated non-linear prediction.    
   
   
       25 . An article of manufacture having one or more computer readable media storing instructions thereon which, when executed by a system, cause the system to perform a method comprising: 
 performing motion compensation to generate a motion compensated prediction using a block from a previously coded frame;    performing non-linear filtering on the motion compensated prediction in the transform domain with a non-linear filter as part of a fractional interpolation process to generate motion compensated non-linear prediction;    subtracting the motion compensated non-linear prediction from a block in a current frame to produce a residual frame; and    coding the residual frame.    
   
   
       26 . The article of manufacture defined in  claim 25  wherein the method further comprises applying non-linear filtering to a previously decoded frame, and wherein performing motion compensation comprises generating the motion compensated prediction using the non-linearly filtered previously decoded frame.  
   
   
       27 . A method comprising: 
 generating a predicted block using a block from a previously decoded frame;    performing non-linear filtering on the predicted block in the transform domain with a non-linear prediction filter as part of a fractional interpolation process to generate a non-linear prediction;    decoding a residual frame; and    adding the residual frame to the non-linear prediction.    
   
   
       28 . The method defined in  claim 27  wherein non-linear filtering is based on prediction parameters.  
   
   
       29 . The method defined in  claim 28  further comprising determine the non-linear filter based on the prediction parameters.  
   
   
       30 . The method defined in  claim 29  further comprising: 
 applying a transform to the predicted block to obtain a first set of transform coefficients; thresholding each transform coefficient based on a threshold to identify a set of one or more transform coefficients to change;    modifying the first set of transform coefficients to create a second set of transform coefficients by changing transform coefficients in the set of one or more transform coefficients; and    applying an inverse transform to the second set of transform coefficients to generate the non-linear prediction.    
   
   
       31 . The method defined in  claim 26  further comprising: 
 applying a transform to the predicted block to obtain a first set of transform coefficients;    ranking transform coefficients based on magnitude to identify a set of one or more transform coefficients to change;    modifying the first set of transform coefficients to create a second set of transform coefficients by changing transform coefficients in the set of one or more transform coefficients; and    applying an inverse transform to the second set of transform coefficients to generate the non-linear prediction.    
   
   
       32 . The method defined in  claim 27  further comprising: 
 applying a transform to the predicted block to obtain a first set of transform coefficients;    percentage filtering each transform coefficient of the block to identify a set of one or more transform coefficients to change;    modifying the first set of transform coefficients to create a second set of transform coefficients by changing transform coefficients in the set of one or more transform coefficients; and    applying an inverse transform to the second set of transform coefficients to generate the non-linear prediction.    
   
   
       33 . The method defined in  claim 27  further comprising: 
 applying a transform to the predicted block to obtain a first set of transform coefficients;    assigning an index to each transform coefficient in the block;    identifying a set of one or more transform coefficients to change based on the indices;    modifying the first set of transform coefficients to create a second set of transform coefficients by changing transform coefficients in the set of one or more transform coefficients; and    applying an inverse transform to the second set of transform coefficients to generate the non-linear prediction.    
   
   
       34 . The method defined in  claim 27  further comprising: 
 applying a transform to the predicted block to obtain a first set of transform coefficients;    scaling one or more transform coefficients to create a second set of transform coefficients; and    applying an inverse transform to the second set of transform coefficients to generate the non-linear prediction.    
   
   
       35 . An apparatus comprising: 
 a motion compensation unit to generate a predicted block using a block from a previously decoded frame;    a non-linear prediction filter to perform non-linear filtering on the predicted block in the transform domain with a non-linear prediction filter as part of a fractional interpolation process to generate a non-linear prediction;    a decoder to decode a residual frame; and    an adder to add the residual frame to the non-linear prediction.    
   
   
       36 . The apparatus defined in  claim 35  wherein the non-linear prediction filter comprises: 
 a forward transform module to apply a transform to the predicted block to obtain a first set of transform coefficients;    a module to perform thresholding on each transform coefficient based on a threshold to identify a set of one or more transform coefficients to change and to modify the first set of transform coefficients to create a second set of transform coefficients by changing transform coefficients in the set of one or more transform coefficients; and    an inverse transform module to apply an inverse transform to the second set of transform coefficients to generate the non-linear prediction.    
   
   
       37 . The apparatus defined in  claim 35  wherein the non-linear prediction filter comprises: 
 a forward transform module to apply a transform to the predicted block to obtain a first set of transform coefficients;    a module to sort transform coefficients based on magnitude to identify a set of one or more transform coefficients to change and modify the first set of transform coefficients to create a second set of transform coefficients by changing transform coefficients in the set of one or more transform coefficients; and    an inverse transform module to apply an inverse transform to the second set of transform coefficients to generate the non-linear prediction.    
   
   
       38 . The apparatus defined in  claim 35  wherein the non-linear prediction filter comprises: 
 a forward transform module to apply a transform to the predicted block to obtain a first set of transform coefficients;    a module to perform percentage filtering each transform coefficient of the block to identify a set of one or more transform coefficients to change and to modify the first set of transform coefficients to create a second set of transform coefficients by changing transform coefficients in the set of one or more transform coefficients; and    an inverse transform module to apply an inverse transform to the second set of transform coefficients to generate the non-linear prediction.    
   
   
       39 . The apparatus defined in  claim 35  wherein the non-linear prediction filter comprises: 
 a forward transform module to apply a transform to the predicted block to obtain a first set of transform coefficients;    a module to assign an index to each transform coefficient in the block, identify a set of one or more transform coefficients to change based on the indices, and modify the first set of transform coefficients to create a second set of transform coefficients by changing transform coefficients in the set of one or more transform coefficients; and    an inverse transform module to apply an inverse transform to the second set of transform coefficients to generate the non-linear prediction.    
   
   
       40 . The apparatus defined in  claim 35  wherein the non-linear prediction filter comprises: 
 a forward transform module to apply a transform to the predicted block to obtain a first set of transform coefficients;    a module to scale one or more transform coefficients to create a second set of transform coefficients; and    an inverse transform module to apply an inverse transform to the second set of transform coefficients to generate the non-linear prediction.    
   
   
       41 . An article of manufacture having one or more computer readable media storing instructions thereon which, when executed by a system, cause the system to perform a method comprising: 
 generating a predicted block using a block from a previously decoded frame;    performing non-linear filtering on the predicted block in the transform domain with a non-linear prediction filter as part of a fractional interpolation process to generate a non-linear prediction;    decoding a residual frame; and    adding the residual frame to the non-linear prediction.