Method and apparatus for region-based moving image encoding and decoding

Abstract

In partitioning and encoding an image into multiple regions, the degree of freedom of the region shape has generally been low and setting regions based on image features was difficult. A moving image encoding apparatus includes a region partitioning section, an encoder, and a memory for motion-compensated prediction. The region partitioning section includes a partitioning processing section and a integration processing section. The partitioning processing section partitions the input image based on a criterion relating to the state of partition. The integration processing section integrates mutually close regions based on a criterion relating to the state of integration. Thereafter, each region is encoded. A large variety of region shapes can be produced by the integration processing section.

1 . A moving image encoding method including steps of: 
 partitioning an input image into a plurality of regions based on a predetermined partition judgment criterion;    integrating regions with adjacent regions, for the various plurality of partitioned regions, based on a predetermined integration judgment criterion; and    encoding image signals for regions remaining after integration.    
   
   
       2 . A method as in  claim 1  wherein said partition judgment criterion involves a comparison between performing encoding with a given region partitioned and performing encoding without the region partitioned.  
   
   
       3 . A method as in  claim 1  wherein said integration judgment criterion involves a comparison between performing encoding with a given region composed with adjacent regions and performing encoding without the region composed with the adjacent regions.  
   
   
       4 . A moving image encoding apparatus including: 
 a region partitioning section which includes a partitioning processing section for partitioning an input image into a plurality of regions based on a predetermined partition judgment criterion and an integration processing section for integrating each of a plurality of regions partitioned by the partitioning processing section with adjacent regions; and    an encoder for encoding image signals for each of the regions remaining after integration by the integration processing section.    
   
   
       5 . An apparatus as in  claim 4  wherein said integration processing section comprises: 
 a provisional encoder for preliminarily encoding an image per region and calculating the amount of code thereof;    a decoder for decoding the image encoded by the provisional encoder;    an encoding distortion calculating section for calculating encoding distortion by using the image decoded by the decoder; and    an evaluation value calculating section for calculating an evaluation value for judging merit of encoding while taking into consideration both the amount of code and the encoding distortion;    wherein it is determined for each region whether or not to perform integration for the region based on a result of comparing the evaluation value that is obtained in the case where the region is integrated with adjacent regions and the evaluation is value that is obtained in the case where the region is not integrated with adjacent regions.    
   
   
       6 . An apparatus as in  claim 4  wherein said partitioning processing section includes: 
 an activity calculating section for calculating prediction error power accompanying motion-compensated prediction of each region as an activity of the region; and    a partitioning judgment section for comparing the calculated activity with a criterion value that was set in advance;    and further partitions regions having activity greater than the criterion value into smaller regions.    
   
   
       7 . An apparatus as in  claim 4  wherein said partitioning processing section includes: 
 an activity calculating section for calculating edge intensity of an original signal for each region as the activity of the region; and    a partitioning judgment section for comparing the calculated activity with a criterion value that was set in advance;    and further partitions regions having activity greater than the criterion value into smaller regions.    
   
   
       8 . An apparatus as in  claim 4  wherein said partitioning processing section includes: 
 an activity calculating section for calculating, for each region, a linear sum of a plurality of numeric values indicating the characteristics of the image of the region; and    a partitioning judgment section for comparing the calculated activity with a criterion value that was set in advance;    and further partitions regions having activity greater than the criterion value into smaller regions.    
   
   
       9 . An apparatus as in  claim 8  wherein said plurality of numeric values includes a prediction error power and a motion parameter of each region which accompany motion-compensated prediction.  
   
   
       10 . An apparatus as in  claim 8  wherein said plurality of numeric values includes amount of code of a motion parameter of each region, a prediction error power which accompanies motion compensation, a dispersion value of an original signal, edge intensity, and magnitude of the motion parameter of each region.  
   
   
       11 . An apparatus as in one of claims  6  wherein said partitioning processing section further includes a class identifying section and judges whether or not to partition each region on the basis of both said activity and class.  
   
   
       12 . An apparatus as in  claim 11  wherein said class identifying section observes an object structure spanning a plurality of regions and decides classes for the regions.  
   
   
       13 . An apparatus as in  claim 12  wherein said object structure is judged on the basis of original signal dispersion of the region, edge intensity, and degree of connection of the edge with adjacent regions.  
   
   
       14 . An apparatus as in  claim 11  wherein said class identifying section observes features of an image; performs detection of objects; and, based on the results thereof, decides classes for the regions.  
   
   
       15 . An apparatus as in  claim 14  wherein said class identifying section stores in advance, for each object predicted to be included in the image, features of the image including the object, and determines the class of each region based on degree of coincidence of the features of the image of each region and the stored features of the object.  
   
   
       16 . An apparatus as in  claim 4  wherein said partitioning processing section includes: 
 a provisional encoder for preliminarily encoding the image for each region and calculating the amount of code thereof;    a decoder for decoding the image encoded by the provisional encoder;    an encoding distortion calculating section for calculating an encoding distortion using the image that was decoded by the decoder; and    an evaluation value calculating section for calculating the evaluation value for judging merit of encoding while taking into consideration both the amount of code and the encoding distortion;    wherein it is determined for each region whether or not to perform partitioning for the region based on a result comparing the evaluation value that is obtained in the case where the region is further partitioned into smaller regions and the evaluation value that is obtained in the case where the region is not further partitioned into smaller regions.    
   
   
       17 . An apparatus as in  claim 5  wherein a quantization parameter of a prediction error signal accompanying motion-compensated prediction is variably set in said provisional encoder, and said evaluation value calculating section calculates the evaluation value while varying the quantization parameter.  
   
   
       18 . An apparatus as in  claim 5  wherein an evaluation value calculating section for obtaining as an evaluation value a linear sum of the prediction error power and the amount of code of motion parameter of each region accompanying motion-compensated prediction is disposed in a stage prior to that of said provisional encoder, and said provisional encoder detects the motion parameter based on the evaluation value.  
   
   
       19 . An apparatus for inputting and decoding encoded data of an image that was encoded after being partitioned into a plurality of regions, comprising: 
 a region shape restoring section for restoring, based on region shape information included in the encoded data, the shape of each region that was partitioned during encoding; and    an image data decoder for decoding the image of each region from encoded data corresponding to its region.    
   
   
       20 . An apparatus of  claim 19  wherein said region shape information includes information relating to processing when regions are partitioned and integrated during encoding, and, based on this information, said region shape restoring section identifies the partitioned state of regions by reproducing the same process as that of the encoding apparatus.  
   
   
       21 . A method for encoding plurality of pre-partitioned regions composing an image signal, said method comprising steps of: 
 partitioning at least one region of the pre-partitioned plurality of regions based on a partition judgment criterion, said partitioning converting the pre-partitioned regions into a first plurality of regions having at least two unequal area regions;    combining at least two regions from said first plurality of regions based on an integration judgment criterion that is image related, said combining converting said first plurality of regions into a second plurality of regions; and    encoding separately regions of said second plurality of regions.    
   
   
       22 . An apparatus for encoding plurality of pre-partitioned regions composing an image signal, said apparatus comprising: 
 a processor partitioning at least one region of the pre-partitioned plurality of regions based on a partition judgment criterion to convert the pre-partitioned regions into a first plurality of regions having at least two unequal area regions;    a processor combining at least two regions from said first plurality of regions based on an integration judgment criterion that is image related to convert said first plurality of regions into a second plurality of regions; and    an encoder separately encoding regions of said second plurality of regions.    
   
   
       23 . A method for encoding an image signal, said method comprising: partitioning an image signal into a first plurality of regions; partitioning at least one region of said first plurality of regions based on a partition judgment criterion, said partitioning at least one region converting said first plurality of regions into a second plurality of regions having at least two unequal area regions; combining at least two regions from said second plurality of regions based on an integration judgment criterion, said combining at least two regions converting said second plurality of regions into a third plurality of regions; and encoding separately regions of said third plurality of regions.  
   
   
       24 . A method as in  claim 23 , wherein said at least two regions are contiguous.  
   
   
       25 . A method as in  claim 24 , wherein said at least two regions are in contact.  
   
   
       26 . A method as in  claim 25 , wherein said at least two regions share a boundary segment.  
   
   
       27 . A method as in  claim 23 , wherein said step of partitioning at least one region includes iteratively partitioning said at least one region based on said partition judgment criterion.  
   
   
       28 . A method as in  claim 23 , wherein said partitioning judgement criterion includes comparing between a result of said step of encoding when said at least one region is partitioned with a result of said step of encoding when said at least one region is not partitioned.  
   
   
       29 . A method as in  claim 23 , wherein said integration judgement criterion includes comparing between a result of said step of encoding when said at least two regions are combined with a result of said encoding when said at least two regions are not combined.  
   
   
       30 . A method as in  claim 29 , wherein said integration judgement criterion includes comparing between a result of said step of encoding when said at least two regions, are combined with a result of said encoding when said at least two regions are not combined.  
   
   
       31 . An apparatus for encoding an image signal, said apparatus comprising: a processor partitioning an image signal into a first plurality of regions; a processor partitioning at least one region of said first plurality of regions based on a partition judgment criterion to convert said first plurality of regions into a second plurality of regions having at least two unequal area regions; a processor combining at least two regions from said second plurality of regions based on an integration judgment criterion to convert said second plurality of regions into a third plurality of regions; and a processor encoding separately regions of said third plurality of regions.  
   
   
       32 . An apparatus as in  claim 31 , wherein said at least two regions are contiguous.  
   
   
       33 . An apparatus as in  claim 32 , wherein said at least two regions are in contact.  
   
   
       34 . An apparatus as in  claim 33 , wherein said at least two regions share a boundary segment.  
   
   
       35 . An apparatus as in  claim 31 , wherein said processor partitioning at least one region iteratively partitions said at least one region based on said partition judgment criterion.  
   
   
       36 . An apparatus as in  claim 31 , wherein said partitioning judgement criterion includes comparing between a result of encoding when said at least one region is partitioned with a result of encoding when said at least one region is not partitioned.  
   
   
       37 . An apparatus as in  claim 31 , wherein said integration judgement criterion includes comparing between a result of encoding when said at least two regions are combined with a result of encoding when said at least two regions are not combined.  
   
   
       38 . An apparatus as in  claim 37 , wherein said integration judgement criterion includes comparing between a result of encoding when said at least two regions are combined with a result of said encoding when said at least two regions are not combined.