Techniques for predicting perceptual video quality

Abstract

In one embodiment of the present invention, a quality trainer and quality calculator collaborate to establish a consistent perceptual quality metric via machine learning. In a training phase, the quality trainer leverages machine intelligence techniques to create a perceptual quality model that combines objective metrics to optimally track a subjective metric assigned during viewings of training videos. Subsequently, the quality calculator applies the perceptual quality model to values for the objective metrics for a target video, thereby generating a perceptual quality score for the target video. In this fashion, the perceptual quality model judiciously fuses the objective metrics for the target video based on the visual feedback processed during the training phase. Since the contribution of each objective metric to the perceptual quality score is determined based on empirical data, the perceptual quality score is a more accurate assessment of observed video quality than conventional objective metrics.

What is claimed is: 
     
       1. A computer-implemented method, comprising:
 selecting a set of objective metrics that includes an anti-noise signal-to-noise ratio; 
 for each training video included in a set of training videos, receiving a subjective value for a perceptual video quality metric and a set of objective values for the set of objective metrics, wherein the subjective value and the set of objective values describe the training video; 
 deriving a composite relationship based on a correlation between the subjective value, the set of objective values, and a measure of pixel motion within at least one of the set of training videos, wherein the composite relationship specifies a level of contribution for at least one of the set of objective metrics to the perceptual video quality metric; 
 for a target video, calculating a first set of values for the set of objective metrics; and 
 applying the composite relationship to the first set of values to generate an output value for the perceptual video quality metric. 
 
     
     
       2. The computer-implemented method of  claim 1 , wherein deriving the composite relationship comprises performing one or more training operations on the data sets. 
     
     
       3. The computer-implemented method of  claim 2 , wherein performing one or more training operations on a given data set comprises applying a support vector machine algorithm or an artificial neural network algorithm to the set of objective values included in the data set. 
     
     
       4. The computer-implemented method of  claim 1 , further comprising:
 determining that a value included in the first set of values exceeds a predetermined threshold; and 
 modifying the output value for the perceptual quality metric based on an adjustment factor. 
 
     
     
       5. The computer-implemented method of  claim 1 , further comprising:
 computing a motion value based on pixel differences between two consecutive frames of the target video; 
 determining that the motion value exceeds a predetermined threshold; and 
 increasing the output value for the perceptual quality metric by a predetermined amount. 
 
     
     
       6. The computer-implemented method of  claim 1 , wherein the set of objective metrics further includes at least one of a detail loss measure and a visual information fidelity. 
     
     
       7. The computer-implemented method of  claim 1 , wherein the target video is derived from a source video, and calculating a first set of values for the set of objective metrics comprises calculating a first value for the anti-noise signal-to-noise ratio. 
     
     
       8. The computer-implemented method of  claim 7 , wherein calculating a first value for the anti-noise signal-to-noise ratio comprises:
 applying a first low pass filter to the source video; 
 applying a second low pass filter to the target video that is stronger than the first low pass filter; and 
 performing one or more signal-to-noise ratio calculations based on the filtered source video and the filtered target video. 
 
     
     
       9. The computer-implemented method of  claim 1 , wherein a first subjective value for the perceptual video quality metric is a human-observed score for the visual quality of a reconstructed video that is derived from the first training video. 
     
     
       10. The computer-implemented method of  claim 1 , wherein a first training video included in the set of training videos includes at least one of compressed data and scaled data. 
     
     
       11. One or more non-transitory computer-readable media including instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of:
 selecting a set of objective metrics that includes an anti-noise signal-to-noise ratio; 
 for each training video included in a set of training videos, receiving a subjective value for a perceptual video quality metric and a set of objective values for the set of objective metrics, wherein the subjective value and the set of objective values describe the training video; 
 deriving a composite relationship based on a correlation between the subjective value and the set of objective values, wherein the composite relationship specifies a level of contribution for at least one of the set of objective metrics to the perceptual video quality metric; 
 for a target video, calculating a first set of values for the set of objective metrics; and 
 applying the composite relationship to the first set of values to generate an output value for the perceptual video quality metric. 
 
     
     
       12. The non-transitory computer-readable medium of  claim 11 , wherein deriving the composite relationship comprises performing one or more training operations on the data sets. 
     
     
       13. The non-transitory computer-readable medium of  claim 12 , wherein performing one or more training operations on a given data set comprises applying a support vector machine algorithm or an artificial neural network algorithm to the set of objective values included in the data set. 
     
     
       14. The non-transitory computer-readable medium of  claim 11 , further comprising:
 determining that a value included in the first set of values exceeds a predetermined threshold; and 
 modifying the output value for the perceptual quality metric based on an adjustment factor. 
 
     
     
       15. The non-transitory computer-readable medium of  claim 11 , further comprising:
 computing a motion value based on pixel differences between two consecutive frames of the target video; 
 determining that the motion value exceeds a predetermined threshold; and 
 increasing the output value for the perceptual quality metric by a predetermined amount. 
 
     
     
       16. The non-transitory computer-readable medium of  claim 11 , wherein the set of objective metrics further includes at least one of a detail loss measure and a visual information fidelity. 
     
     
       17. The non-transitory computer-readable medium of  claim 11 , wherein the target video is derived from a source video, and calculating a first set of values for the set of objective metrics comprises calculating a first value for the anti-noise signal-to-noise ratio. 
     
     
       18. The non-transitory computer-readable medium of  claim 17 , wherein calculating a first value for the anti-noise signal-to-noise ratio comprises:
 applying a first low pass filter to the source video; 
 applying a second low pass filter to the target video that is stronger than the first low pass filter; and 
 performing one or more signal-to-noise ratio calculations based on the filtered source video and the filtered target video. 
 
     
     
       19. The non-transitory computer-readable medium of  claim 11 , wherein a first subjective value for the perceptual video quality metric is a human-observed score for the visual quality of a reconstructed video that is derived from the first training video. 
     
     
       20. The non-transitory computer-readable medium of  claim 11 , wherein a first training video included in the set of training videos includes at least one of compressed data and scaled data. 
     
     
       21. A system, comprising:
 one or more memories including instructions; and 
 one or more processors that are coupled to the one or more memories and, when executing the instructions, are configured to perform the steps of:
 selecting a set of one or more objective metrics representing video characteristics; 
 for each training video included in a set of training videos, receiving a subjective value for a perceptual video quality metric and a set of one or more objective values for the set of one or more objective metrics, wherein the subjective value and the set of objective values describe the training video; 
 deriving a composite relationship based on a correlation between the subjective value and the set of one or more objective values, wherein the composite relationship specifies a level of contribution for at least one of the set of one or more objective metrics to the perceptual video quality metric; 
 for a target video, calculating a first set of one or more values for the set of one or more objective metrics; and 
 applying the composite relationship to the first set of one or more values to generate an output value for the perceptual video quality metric.