Method and apparatus for adaptive encoding framed data sequences

Abstract

Methods and apparatus for adaptive encoding of at least a part of a current frame of a sequence of frames of framed data are described which operate on a block-by-block coding basis. The methods and apparatus divide at least a part of the current frame into blocks and then perform a first sub-encoding step on a block. Thereafter a second sub-encoding step is performed on the first sub-encoded block whereby the second sub-encoding step is optimized by adapting its encoding parameters based on a quantity of the first sub-encoded part of the current frame. The quantity is determined by prediction from a reference frame. Then the same steps are performed on another block of the part of the current frame. Typically, the framed data will be video frames for transmission over a transmission channel. The adaptation of the parameters for the second sub-encoding step may be made dependent upon the characteristics or limitations, e.g. bandwidth limitation, of the channel. In addition, the current frame may be discarded based on the predicted quantity and/or based on fullness of a buffer.

What is claimed is: 
     
       1. A method of adaptive encoding at least a part of a current frame of a sequence of frames of framed data, with respect to a reference frame comprised in the sequence, the method comprising:
 dividing the reference frame into blocks and identifying selected blocks of the reference frame in accordance with the performance of a first sub-encoding that is applied to the reference frame; 
 computing a characteristic based on the identifying of the blocks and from the frames of the sequence only those frames that are a reference frame, wherein the identifying of the blocks depends upon motion vectors determined for the blocks, and wherein the blocks of the reference frame have a first identity when the blocks are intra-coded or when the blocks have a substantial zero motion vector, the blocks of the reference frame have a second identity otherwise, the computed characteristic being based on:
 measures of prediction errors of blocks with the first identity; and 
 measures of prediction errors of blocks with a second identity combined with the time elapsed between the current frame and the reference frame; 
 
 performing the first sub-encoding on the current frame; and 
 performing a second sub-encoding on the first sub-encoded current frame, the second sub-encoding adapting at least one encoding parameter based at least in part upon the computed characteristic. 
 
     
     
       2. The method of  claim 1 , wherein the characteristic is a sum of at least:
 the sum of all measures of prediction errors of blocks that have the first identity; and 
 a normalized sum of all measures of prediction errors of blocks that have the second identity multiplied by the time elapsed between the current frame and the reference frame. 
 
     
     
       3. A method of adaptive encoding at least a part of a current frame of a sequence of frames of framed data, with respect to a reference frame comprised in the sequence, the method comprising:
 dividing the reference frame into blocks, wherein the blocks of the reference frame have a first identity when the blocks are intra-coded or when the blocks have a substantial zero motion vector, the blocks of the reference frame have a second identity otherwise, computing a characteristic which is based on:
 measures of prediction errors of blocks that have the first identity, and 
 measures of prediction errors of blocks that have the second identity combined the time elapsed between the current frame and the reference frame; 
 
 identifying selected blocks of the reference frame in accordance with the performance of a first sub-encoding that is applied to the reference frame; 
 computing a characteristic based on the identity of the blocks; 
 performing the first sub-encoding on the current frame; and 
 performing a second sub-encoding on the first sub-encoded current frame, wherein the second sub-encoding adapts at least one encoding parameter based on the computed characteristic. 
 
     
     
       4. The method of  claim 3 , wherein the characteristic is a sum of at least:
 the sum of all measures of prediction errors of blocks that have the first identity; and 
 a normalized sum of all measures of prediction errors of blocks that have the second identity multiplied by the time elapsed between the current frame and the reference frame. 
 
     
     
       5. A method of adaptive encoding at least a part of a current frame of a sequence of frames of framed data, with respect to a reference frame comprised in the sequence, the method comprising:
 dividing the reference frame into blocks and identifying selected blocks of the reference frame in accordance with the performance of a first sub-encoding that is applied to the reference frame, wherein the identifying selected blocks depends upon motion vectors determined for the blocks; 
 computing a characteristic indicative of an energy content, the characteristic being determined by prediction without using a result of the first sub-encoding of the current frame, without using blocks of the current frame, and based on the identifying of the blocks and from the frames of the sequence only those frames that are a reference frame, wherein computing of the characteristic is based upon the time elapsed between the current frame and the reference frame or frames; 
 performing the first sub-encoding on the current frame; and 
 performing a second sub-encoding on the first sub-encoded current frame, the second sub-encoding adapting at least one encoding parameter based at least in part upon the computed characteristic, wherein the blocks of the reference frame have a first identity when the blocks are intra-coded or when the blocks have a substantial zero motion vector, the blocks of the reference frame have a second identity otherwise, the computed characteristic being the sum of:
 the sum of all measures of prediction errors of blocks with the first identity; and 
 a normalized sum of all measures of prediction errors of blocks with the second identity multiplied with the time elapsed between the current frame and the reference frame. 
 
 
     
     
       6. A method of adaptive encoding at least a part of a current frame of a sequence of frames of framed data, with respect to a reference frame comprised in the sequence, the method comprising:
 dividing the reference frame into blocks;   identifying selected blocks of the reference frame in accordance with the performance of a first sub-encoding that is applied to the reference frame;   performing the first sub-encoding on the current frame;   computing a characteristic indicative of an energy content based on the identifying of the blocks and from the frames of the sequence only those frames that are a reference frame, wherein the characteristic is determined by prediction without using the first sub-encoded frame and without using blocks of the current frame, wherein computing of the characteristic is based upon the time elapsed between the current frame and the reference frame or frames, and wherein the computing of the characteristic is derived at least in part by calculating the time elapsed between the current frame and the reference frame wherein the blocks of the reference frame have a first identity when the blocks are intra-coded or when the blocks have a substantial zero motion vector, the blocks of the reference frame have a second identity otherwise, the computed characteristic being the sum of:
 the sum of all measures of prediction errors of blocks that have the first identity; and 
 a normalized sum of all measures of prediction errors of blocks that have the second identity multiplied by the time elapsed between the current frame and the reference frame; 
   performing the first sub-encoding on the current frame; and   performing a second sub-encoding on the first sub-encoded frame, wherein the second sub-encoding adapts at least one encoding parameter based on the computed characteristic.   
     
     
       7. The method of claim 5, wherein performing the first sub-encoding on the current frame comprises performing the first sub-encoding on a set of blocks of the current frame, and performing the second sub-encoding comprises performing the second sub-encoding on the first sub-encoded set of blocks of the current frame before performing first and second sub-encoding of another set of blocks of the current frame.  
     
     
       8. The method of claim 7, wherein the set of blocks comprises one block.  
     
     
       9. The method of claim 5 further comprising transmitting the second sub-encoded frame over a wireless channel, and wherein the second sub-encoding adapts at least one encoding parameter based at least in part upon a characteristic of the wireless channel.  
     
     
       10. The method of claim 5, wherein the second sub-encoding comprises wavelet encoding, quadtree encoding, binary tree encoding, discrete cosine transform (DCT) encoding, and matching pursuits coding.  
     
     
       11. The method of claim 5, wherein the framed data is video data.  
     
     
       12. The method of claim 5, wherein the framed data is 3-D video data.  
     
     
       13. The method of claim 5, wherein the content characteristic comprises energy content, information content, and content complexity.  
     
     
       14. The method of claim 5, wherein the characteristic indicative of an energy content is determined prior to buffering the current frame.  
     
     
       15. A method of adaptive encoding at least a part of a current frame of a sequence of frames of framed data, with respect to a reference frame comprised in the sequence, the method comprising:
 dividing the reference frame into blocks and identifying selected blocks of the reference frame in accordance with the performance of a first sub-encoding that is applied to the reference frame;   computing a characteristic indicative of an energy content based on the identifying of the blocks in a first identity and a second identity and from the frames of the sequence only those frames that are a reference frame, the computed characteristic being based on measures of prediction errors of blocks with the first identity and the second identity;   performing the first sub-encoding on the current frame; and   performing a second sub-encoding on the first sub-encoded current frame, the second sub-encoding adapting at least one encoding parameter based at least in part upon the computed characteristic.    
     
     
       16. A method of adaptive encoding at least a part of a current frame of a sequence of frames of framed data, with respect to a reference frame comprised in the sequence, the method comprising:
 dividing the reference frame into blocks and identifying selected blocks of the reference frame in accordance with the performance of a first sub-encoding that is applied to the reference frame;   computing a characteristic based on the identifying of the blocks and from the frames of the sequence only those frames that are a reference frame, wherein the blocks of the reference frame have a first identity when the blocks are intra-coded or when the blocks have a substantial zero motion vector, the blocks of the reference frame have a second identity otherwise, the computed characteristic being based on:
 measures of prediction errors of blocks with the first identity; and 
 measures of prediction errors of blocks with a second identity combined with the time elapsed between the current frame and the reference frame; 
   performing the first sub-encoding on the current frame; and   performing a second sub-encoding on the first sub-encoded current frame, the second sub-encoding adapting at least one encoding parameter based at least in part upon the computed characteristic.    
     
     
       17. A method of adaptive encoding at least a part of a current frame of a sequence of frames of framed data, with respect to a reference frame comprised in the sequence, the method comprising:
 dividing the reference frame into blocks and identifying selected blocks of the reference frame in accordance with the performance of a first sub-encoding that is applied to the reference frame;   computing a characteristic indicative of an energy content, the characteristic being determined by prediction without using a result of the first sub-encoding of the current frame, without using blocks of the current frame, and based on the identifying of the blocks and from the frames of the sequence only those frames that are a reference frame, wherein computing of the characteristic is based upon the time elapsed between the current frame and the reference frame or frames;   performing the first sub-encoding on the current frame; and   performing a second sub-encoding on the first sub-encoded current frame, the second sub-encoding adapting at least one encoding parameter based at least in part upon the computed characteristic, wherein the blocks of the reference frame have a first identity when the blocks are intra-coded or when the blocks have a substantial zero motion vector, the blocks of the reference frame have a second identity otherwise, the computed characteristic being the sum of:
 the sum of all measures of prediction errors of blocks with the first identity; and 
 a normalized sum of all measures of prediction errors of blocks with the second identity multiplied with the time elapsed between the current frame and the reference frame.  
   
     
     
       18. A method of adaptive encoding at least a part of a current frame of a sequence of frames of framed data, with respect to a reference frame comprised in the sequence, the method comprising:
 dividing the reference frame into blocks;   identifying selected blocks of the reference frame in accordance with the performance of a first sub-encoding that is applied to the reference frame;   computing a characteristic indicative of an energy content based on the identifying of the blocks and from the frames of the sequence only those frames that are a reference frame, wherein the characteristic is determined by prediction without using the first sub-encoded frame and without using blocks of the current frame, wherein computing of the characteristic is based upon the time elapsed between the current frame and the reference frame or frames, and wherein the computing of the characteristic is derived at least in part by calculating the time elapsed between the current frame and the reference frame wherein the blocks of the reference frame have a first identity when the blocks are intra-coded or when the blocks have a substantial zero motion vector, the blocks of the reference frame have a second identity otherwise, the computed characteristic being the sum of:
 the sum of all measures of prediction errors of blocks that have the first identity; and 
 a normalized sum of all measures of prediction errors of blocks that have the second identity multiplied by the time elapsed between the current frame and the reference frame; 
   performing the first sub-encoding on the current frame; and   performing a second sub-encoding on the first sub-encoded current frame, wherein the second sub-encoding adapts at least one encoding parameter based on the computed characteristic.    
     
     
       19. An apparatus for adaptive encoding a sequence of frames of framed data, the frames comprising a current frame and a reference frame, the apparatus comprising:
 a division circuit configured to divide each frame into a plurality of blocks and identify selected blocks of the reference frame;   a decision circuit configured to determine a characteristic indicative of an energy content based on the identification of the blocks into a first identity and a second identity and only from the frames of the sequence that are a reference frame, the determined characteristic being based on measures of prediction errors of blocks according to each identity;   a first sub-encoding circuit configured to encode each frame; and   a second sub-encoding circuit configured to adaptively encode the one or more blocks based at least in part on the characteristic.    
     
     
       20. A non-transitory computer readable storage medium comprising instructions that, when executed by a processor, cause the processor to adaptively encode at least a part of a current frame of a sequence of frames of framed data, with respect to a reference frame comprised in the sequence by:
 dividing the reference frame into blocks and identifying selected blocks of the reference frame in accordance with the performance of a first sub-encoding that is applied to the reference frame;   computing a characteristic indicative of an energy content based on the identifying of the blocks in a first identity and a second identity and from the frames of the sequence only those frames that are a reference frame, the computed characteristic being based on measures of prediction errors of blocks with the first identity and the second identity;   performing the first sub-encoding on the current frame; and   performing a second sub-encoding on the first sub-encoded current frame, the second sub-encoding adapting at least one encoding.