Fractional interpolation for hardware-accelerated video decoding

Abstract

Video decoding innovations for multithreading implementations and graphics processor unit (“GPU”) implementations are described. For example, for multithreaded decoding, a decoder uses innovations in the areas of layered data structures, picture extent discovery, a picture command queue, and/or task scheduling for multithreading. Or, for a GPU implementation, a decoder uses innovations in the areas of inverse transforms, inverse quantization, fractional interpolation, intra prediction using waves, loop filtering using waves, memory usage and/or performance-adaptive loop filtering. Innovations are also described in the areas of error handling and recovery, determination of neighbor availability for operations such as context modeling and intra prediction, CABAC decoding, computation of collocated information for direct mode macroblocks in B slices, reduction of memory consumption, implementation of trick play modes, and picture dropping for quality adjustment.

1 .- 20 . (canceled) 
     
     
         21 . A method of video decoding using a hardware-accelerated video decoder, the method comprising:
 classifying plural blocks according to plural motion vector types, wherein the plural motion vector types differ in terms of complexity of sample value interpolation; and   with a graphics processing unit, performing motion compensation operations for the plural blocks in plural passes corresponding to the plural motion vector types, respectively.   
     
     
         22 . The method of  claim 21  wherein each of the plural motion vector types is associated with a quantum of work for the motion vector type, and wherein the quantum of work for each of the plural motion vector types is 8×8 block. 
     
     
         23 . The method of  claim 21  wherein plural motion vectors for the plural blocks are applied for 4×4 blocks in the motion compensation operations. 
     
     
         24 . The method of  claim 21  wherein the plural motion vector types are:
 an integer motion vector type that represents motion vectors with offsets at integer sample positions; 
 a center offset motion vector type that represents motion vectors with offsets at fractional sample positions whose sample values depend on sample values at sample positions with ½-pixel horizontal offset and ½-pixel vertical offset; and 
 an off-center offset motion vector type that represents motion vectors with offsets at other fractional sample positions. 
 
     
     
         25 . The method of  claim 21  wherein the plural passes include a first pass for an integer motion vector type, followed by a second pass for a center offset motion vector type, followed by a third pass for an off-center offset motion vector type. 
     
     
         26 . The method of  claim 21  wherein, for a first pass of the plural passes, the motion compensation operations include fetching sample values from one or more reference pictures in memory. 
     
     
         27 . The method of  claim 26  wherein the one or more reference pictures for the motion compensation operations are represented as a 3D texture. 
     
     
         28 . The method of  claim 21  wherein, for a second pass of the plural passes, a center offset shader implements at least some of the motion compensation operations. 
     
     
         29 . The method of  claim 28  wherein the center offset shader supports fractional interpolation for plural 4×4 blocks in parallel, and wherein for a given 4×4 block of sample values, the center offset shader includes operations for:
 loading a 9×9 block of sample values as a support region for filtering; 
 computing intermediate sample values by filtering loaded sample values in a first direction, wherein at least some of the intermediate sample values are buffered from block-to-block for use in other interpolation operations; and 
 computing final sample values by filtering the intermediate sample values in a second direction. 
 
     
     
         30 . The method of  claim 21  wherein, for a third pass of the plural passes, an off-center offset shader implements at least some of the motion compensation operations. 
     
     
         31 . The method of  claim 30  wherein the off-center offset shader supports fractional interpolation for plural 4×4 blocks in parallel, and wherein for a given 4×4 block of sample values, the off-center offset shader includes operations for:
 loading sample values as a support region for filtering; and 
 computing final sample values by filtering in one or more directions, without buffering intermediate sample values from block-to-block for use in other interpolation operations. 
 
     
     
         32 . A computer system that includes a hardware-accelerated video decoder adapted to perform a method of video decoding, the computer system including:
 a classifier module for classifying plural blocks according to plural motion vector types that differ in terms of complexity of sample value interpolation; and   a graphics processing unit adapted to run as an integer pixel shader, center offset shader or off-center offset shader depending on motion vector type.   
     
     
         33 . The computer system of  claim 32  wherein each of the plural motion vector types is associated with a quantum of work for the motion vector type, and wherein the quantum of work for each of the plural motion vector types is 8×8 block 
     
     
         34 . The computer system of  claim 32  wherein the plural motion vectors are applied for 4×4 blocks in the motion compensation operations. 
     
     
         35 . The computer system of  claim 32  wherein the integer pixel shader includes operations for fetching sample values from one or more reference pictures. 
     
     
         36 . The computer system of  claim 35  wherein the center offset shader includes operations for interpolation to determine sample values at fractional sample positions whose sample values depend on sample values at sample positions with ½-pixel horizontal offset and ½-pixel vertical offset. 
     
     
         37 . The computer system of  claim 36  wherein the center offset shader supports fractional interpolation for plural 4×4 blocks in parallel, and wherein for a given 4×4 block of sample values, the center offset shader includes operations for:
 loading a 9×9 block of sample values as a support region for filtering; 
 computing intermediate sample values by filtering loaded sample values in a first direction, wherein at least some of the intermediate sample values are buffered from block-to-block for use in other interpolation operations; and 
 computing final sample values by filtering the intermediate sample values in a second direction. 
 
     
     
         38 . The computer system of  claim 32  wherein the off-center offset shader includes operations for interpolation to determine sample values at fractional sample positions whose sample values do not depend on sample values at sample positions with ½-pixel horizontal offset and ½-pixel vertical offset. 
     
     
         39 . The computer system of  claim 38  wherein the off-center offset shader supports fractional interpolation for plural 4×4 blocks in parallel, and wherein for a given 4×4 block of sample values, the off-center offset shader includes operations for:
 loading sample values as a support region for filtering; and 
 computing final sample values by filtering in one or more directions, without buffering intermediate sample values from block-to-block for use in other interpolation operations. 
 
     
     
         40 . A computer system that includes a hardware-accelerated video decoder adapted to perform a method of video decoding, the computer system including:
 a graphics processing unit adapted to run as an integer pixel shader, center offset shader or off-center offset shader depending on motion vector type, wherein the center offset pixel shader includes operations for interpolation to determine sample values at fractional sample positions whose sample values depend on sample values at sample positions with ½-pixel horizontal offset and ½-pixel vertical offset, wherein the center offset shader supports fractional interpolation for plural 4×4 blocks in parallel, and wherein for a given 4×4 block of sample values, the center offset shader includes operations for:
 loading a 9×9 block of sample values as a support region for filtering; 
 computing intermediate sample values by filtering loaded sample values in a first direction, wherein at least some of the intermediate sample values are buffered from block-to-block for use in other interpolation operations; and 
 computing final sample values by filtering the intermediate sample values in a second direction.