Systems and methods for adaptive selection of weights for video coding

Abstract

Techniques and systems are provided for processing video data. For example, a current block of a picture of the video data can be obtained for processing by an encoding device or a decoding device. A pre-defined set of weights for template matching based motion compensation are also obtained. A plurality of metrics associated with one or more spatially neighboring samples of the current block and one or more spatially neighboring samples of at least one reference frame are determined. A set of weights are selected from the pre-defined set of weights to use for the template matching based motion compensation. The set of weights is determined based on the plurality of metrics. The template matching based motion compensation is performed for the current block using the selected set of weights.

What is claimed is: 
     
       1. A method of processing video data, the method comprising:
 obtaining a current block of a picture of the video data; 
 obtaining a pre-defined set of weights including pre-defined values for template matching based motion compensation, the pre-defined set of weights comprising at least a first pair of weights and a second pair of weights; 
 applying a first weight of the first pair of weights of the pre-defined set of weights to one or more spatially neighboring samples of a first reference frame, the first weight of the first pair of weights having a first pre-defined value; 
 applying a second weight of the first pair of weights of the pre-defined set of weights to one or more spatially neighboring samples of a second reference frame, the second weight of the first pair of weights having a second pre-defined value; 
 based on applying the first weight of the first pair of weights to the one or more spatially neighboring samples of the first reference frame and applying the second weight of the first pair of weights to the one or more spatially neighboring samples of the second reference frame, determining a first metric associated with one or more spatially neighboring samples of the current block, the one or more spatially neighboring samples of a first reference frame, and the one or more spatially neighboring samples of a second reference frame; 
 applying a first weight of the second pair of weights of the pre-defined set of weights to the one or more spatially neighboring samples of the first reference frame, the first weight of the second pair of weights having a third pre-defined value; 
 applying a second weight of the second pair of weights of the pre-defined set of weights to the one or more spatially neighboring samples of the second reference frame, the second weight of the second pair of weights having a fourth pre-defined value; 
 based on applying the first weight of the second pair of weights to the one or more spatially neighboring samples of the first reference frame and applying the second weight of the second pair of weights to the one or more spatially neighboring samples of the second reference frame, determining a second metric associated with the one or more spatially neighboring samples of the current block, the one or more spatially neighboring samples of the first reference frame, and the one or more spatially neighboring samples of the second reference frame; 
 comparing the first metric and the second metric; 
 determining the first metric is smaller than the second metric; 
 selecting the first pair of weights as a selected pair of weights based on the first metric being smaller than the second metric; and 
 generating a template matching based motion compensation prediction for the current block based on the selected pair of weights. 
 
     
     
       2. The method of  claim 1 , wherein the first metric includes a sum of absolute differences between the one or more spatially neighboring samples of the current block and the one or more spatially neighboring samples of at least the first reference frame; and
 wherein the second metric includes a sum of absolute differences between the one or more spatially neighboring samples of the current block and the one or more spatially neighboring samples of at least the second reference frame. 
 
     
     
       3. The method of  claim 1 , wherein the first metric includes a sum of absolute transformed differences between the one or more spatially neighboring samples of the current block and the one or more spatially neighboring samples of at least the first reference frame; and
 wherein the second metric includes a sum of absolute transformed differences between the one or more spatially neighboring samples of the current block and the one or more spatially neighboring samples of at least the second reference frame. 
 
     
     
       4. The method of  claim 1 , wherein the first metric includes a sum of squared errors of prediction between the one or more spatially neighboring samples of the current block and the one or more spatially neighboring samples of at least the first reference frame; and
 wherein the second metric includes a sum of squared errors of prediction between the one or more spatially neighboring samples of the current block and the one or more spatially neighboring samples of at least the second reference frame. 
 
     
     
       5. The method of  claim 1 , wherein no weighting parameter is signaled with the video data. 
     
     
       6. The method of  claim 1 , wherein the template matching based motion compensation includes local illumination compensation (LIC). 
     
     
       7. The method of  claim 1 , wherein the template matching based motion compensation includes weighted prediction (WP). 
     
     
       8. An apparatus comprising:
 a memory configured to store video data; and 
 a processor configured to:
 obtain a current block of a picture of the video data; 
 obtain a pre-defined set of weights including pre-defined values for template matching based motion compensation, the pre-defined set of weights comprising at least a first pair of weights and a second pair of weights; 
 apply a first weight of the first pair of weights of the pre-defined set of weights to one or more spatially neighboring samples of a first reference frame, the first weight of the first pair of weights having a first pre-defined value; 
 apply a second weight of the first pair of weights of the pre-defined set of weights to one or more spatially neighboring samples of a second reference frame, the second weight of the first pair of weights having a second pre-defined value; 
 based on applying the first weight of the first pair of weights to the one or more spatially neighboring samples of the first reference frame and applying the second weight of the first pair of weights to the one or more spatially neighboring samples of the second reference frame, determine a first metric associated with one or more spatially neighboring samples of the current block, the one or more spatially neighboring samples of a first reference frame, and the one or more spatially neighboring samples of a second reference frame; 
 apply a first weight of the second pair of weights of the pre-defined set of weights to the one or more spatially neighboring samples of the first reference frame, the first weight of the second pair of weights having a third pre-defined value; 
 apply a second weight of the second pair of weights of the pre-defined set of weights to the one or more spatially neighboring samples of the second reference frame, the second weight of the second pair of weights having a fourth pre-defined value; 
 based on applying the first weight of the second pair of weights to the one or more spatially neighboring samples of the first reference frame and applying the second weight of the second pair of weights to the one or more spatially neighboring samples of the second reference frame, determine a second metric associated with the one or more spatially neighboring samples of the current block, the one or more spatially neighboring samples of the first reference frame, and the one or more spatially neighboring samples of the second reference frame; 
 compare the first metric and the second metric; 
 determine the first metric is smaller than the second metric; 
 selecting the first pair of weights as a selected pair of weights based on the first metric being smaller than the second metric; and 
 generate a template matching based motion compensation prediction for the current block based on the template matching based motion compensation for the current block using the selected pair of weights. 
 
 
     
     
       9. The apparatus of  claim 8 , wherein the first metric includes a sum of absolute differences between the one or more spatially neighboring samples of the current block and the one or more spatially neighboring samples of at least the first reference frame; and
 wherein the second metric includes a sum of absolute differences between the one or more spatially neighboring samples of the current block and the one or more spatially neighboring samples of at least the second reference frame. 
 
     
     
       10. The apparatus of  claim 8 , wherein the first metric includes a sum of absolute transformed differences between the one or more spatially neighboring samples of the current block and the one or more spatially neighboring samples of at least the first reference frame; and
 wherein the second metric includes a sum of absolute transformed differences between the one or more spatially neighboring samples of the current block and the one or more spatially neighboring samples of at least the second reference frame. 
 
     
     
       11. The apparatus of  claim 8 , wherein the first metric includes a sum of squared errors of prediction between the one or more spatially neighboring samples of the current block and the one or more spatially neighboring samples of at least the first reference frame; and
 wherein the second metric includes a sum of squared errors of prediction between the one or more spatially neighboring samples of the current block and the one or more spatially neighboring samples of at least the second reference frame. 
 
     
     
       12. The apparatus of  claim 8 , wherein no weighting parameter is signaled with the video data. 
     
     
       13. The apparatus of  claim 8 , wherein the template matching based motion compensation includes local illumination compensation (LIC). 
     
     
       14. The apparatus of  claim 8 , wherein the template matching based motion compensation includes weighted prediction (WP). 
     
     
       15. The apparatus of  claim 8 , further comprising:
 a display for displaying the video data. 
 
     
     
       16. The apparatus of  claim 8 , wherein the apparatus comprises a mobile device with a camera for capturing pictures. 
     
     
       17. A non-transitory computer-readable medium having stored thereon instructions that, when executed by one or more processors, cause the one or more processors to:
 obtain a current block of a picture of video data; 
 obtain a pre-defined set of weights including pre-defined values for template matching based motion compensation, the pre-defined set of weights comprising at least a first pair of weights and a second pair of weights; 
 apply a first weight of the first pair of weights of the pre-defined set of weights to one or more spatially neighboring samples of a first reference frame, the first weight of the first pair of weights having a first pre-defined value; 
 apply a second weight of the first pair of weights of the pre-defined set of weights to one or more spatially neighboring samples of a second reference frame, the second weight of the first pair of weights having a second pre-defined value; 
 based on applying the first weight of the first pair of weights to the one or more spatially neighboring samples of the first reference frame and applying the second weight of the first pair of weights to the one or more spatially neighboring samples of the second reference frame, determine a first metric associated with one or more spatially neighboring samples of the current block, the one or more spatially neighboring samples of a first reference frame, and the one or more spatially neighboring samples of a second reference frame; 
 apply a first weight of the second pair of weights of the pre-defined set of weights to the one or more spatially neighboring samples of the first reference frame, the first weight of the second pair of weights having a third pre-defined value; 
 apply a second weight of the second pair of weights of the pre-defined set of weights to the one or more spatially neighboring samples of the second reference frame, the second weight of the second pair of weights having a fourth pre-defined value; 
 based on applying the first weight of the second pair of weights to the one or more spatially neighboring samples of the first reference frame and applying the second weight of the second pair of weights to the one or more spatially neighboring samples of the second reference frame, determine a second metric associated with the one or more spatially neighboring samples of the current block, the one or more spatially neighboring samples of the first reference frame, and the one or more spatially neighboring samples of the second reference frame; 
 compare the first metric and the second metric; 
 determine the first metric is smaller than the second metric; 
 select the first pair of weights as a selected pair of weights based on the first metric being smaller than the second metric; and 
 generate a template matching based motion compensation prediction for the current block based on the selected pair of weights. 
 
     
     
       18. The non-transitory computer-readable medium of  claim 17 , wherein the first metric includes a sum of absolute differences between the one or more spatially neighboring samples of the current block and the one or more spatially neighboring samples of at least the first reference frame; and
 wherein the second metric includes a sum of absolute differences between the one or more spatially neighboring samples of the current block and the one or more spatially neighboring samples of at least the second reference frame. 
 
     
     
       19. The non-transitory computer-readable medium of  claim 17 , wherein the first metric includes a sum of absolute transformed differences between the one or more spatially neighboring samples of the current block and the one or more spatially neighboring samples of at least the first reference frame; and
 wherein the second metric includes a sum of absolute transformed differences between the one or more spatially neighboring samples of the current block and the one or more spatially neighboring samples of at least the second reference frame. 
 
     
     
       20. The non-transitory computer-readable medium of  claim 17 , wherein the first metric includes a sum of squared errors of prediction between the one or more spatially neighboring samples of the current block and the one or more spatially neighboring samples of at least the first reference frame; and
 wherein the second metric includes a sum of squared errors of prediction between the one or more spatially neighboring samples of the current block and the one or more spatially neighboring samples of at least the second reference frame. 
 
     
     
       21. The non-transitory computer-readable medium of  claim 17 , wherein no weighting parameter is signaled with the video data. 
     
     
       22. The non-transitory computer-readable medium of  claim 17 , wherein the template matching based motion compensation includes local illumination compensation (LIC). 
     
     
       23. The non-transitory computer-readable medium of  claim 17 , wherein the template matching based motion compensation includes weighted prediction (WP). 
     
     
       24. An apparatus for processing video data, comprising:
 means for obtaining a current block of a picture of the video data; 
 means for obtaining a pre-defined set of weights including pre-defined values for template matching based motion compensation, the pre-defined set of weights comprising at least a first pair of weights and a second pair of weights; 
 means for applying a first weight of the first pair of weights of the pre-defined set of weights to one or more spatially neighboring samples of a first reference frame, the first weight of the first pair of weights having a first pre-defined value; 
 means for applying a second weight of the first pair of weights of the pre-defined set of weights to one or more spatially neighboring samples of a second reference frame, the second weight of the first pair of weights having a second pre-defined value; 
 means for determining, based on applying the first weight of the first pair of weights to the one or more spatially neighboring samples of the first reference frame and applying the second weight of the first pair of weights to the one or more spatially neighboring samples of the second reference frame, a first metric associated with one or more spatially neighboring samples of the current block, the one or more spatially neighboring samples of a first reference frame, and the one or more spatially neighboring samples of a second reference frame; 
 means for applying a first weight of the second pair of weights of the pre-defined set of weights to the one or more spatially neighboring samples of the first reference frame, the first weight of the second pair of weights having a third pre-defined value; 
 means for applying a second weight of the second pair of weights of the pre-defined set of weights to the one or more spatially neighboring samples of the second reference frame, the second weight of the second pair of weights having a fourth pre-defined value; 
 means for determining, based on applying the first weight of the second pair of weights to the one or more spatially neighboring samples of the first reference frame and applying the second weight of the second pair of weights to the one or more spatially neighboring samples of the second reference frame, a second metric associated with the one or more spatially neighboring samples of the current block, the one or more spatially neighboring samples of the first reference frame, and the one or more spatially neighboring samples of the second reference frame; 
 means for comparing the first metric and the second metric; 
 means for determining the first metric is smaller than the second metric; 
 means for selecting the first pair of weights as a selected pair of weights based on the first metric being smaller than the second metric; and 
 means for generating a template matching based motion compensation prediction for the current block based on the selected pair of weights the template matching based motion compensation for the current block using the selected pair of weights.