Image processing apparatus and method, and storage medium

Abstract

This invention provides an image processing apparatus/method characterized by inputting successive image data, dividing the image data into blocks each formed from a plurality of pixels, detecting the motion vector of each block, calculating a probability at which the block has a predetermined area label, in accordance with the motion vector and the area label of a neighboring block, determining the area label of each block in accordance with the calculation result, and dividing the image data into areas of respective objects in accordance with the determined area label.

What is claimed is:  
     
       1. An image processing apparatus comprising: 
       a) input means for inputting successive image data;  
       b) block division means for dividing the image data into blocks each formed from a plurality of pixels;  
       c) detection means for detecting a motion vector of each block;  
       d) calculation means for calculating a probability at which the block has a predetermined area label, in accordance with the motion vector and an area label of a neighboring block;  
       e) determination means for determining an area label of each block in accordance with an output from said calculation means; and  
       f) area division means for dividing the image data into areas of respective objects in accordance with an output from said determination means.  
     
     
       2. An apparatus according to  claim 1 , wherein said calculation means calculates the probability using a model of a Markov random field. 
     
     
       3. An apparatus according to  claim 1 , wherein said detection means detects the motion vector on the basis of an inter-block matching degree calculated from an inner product of a color vector representing a feature amount of each block. 
     
     
       4. An apparatus according to  claim 1 , wherein said detection means detects the motion vector on the basis of an inter-block matching degree calculated from a norm of a difference vector of a color vector representing a feature amount of each block. 
     
     
       5. An apparatus according to  claim 1 , further comprising means for causing said block division means to re-divide a block at a boundary portion between areas divided by said area division means, and executing processes of said detection means, calculation means, determination means, and area division means for the re-divided block. 
     
     
       6. An apparatus according to  claim 1 , wherein said input means includes image sensing means for sensing a subject image and outputting the serial image data. 
     
     
       7. An apparatus according to  claim 1 , further comprising encoding means for encoding each area divided by said area division means. 
     
     
       8. An apparatus according to  claim 7 , wherein said encoding means executes encoding processing complying with an MPEG-4 standard. 
     
     
       9. An apparatus according to  claim 7 , further comprising recording means for recording image data encoded by said encoding means on a recording medium. 
     
     
       10. An apparatus according to  claim 7 , further comprising transmission means for transmitting image data encoded by said encoding means to an external device. 
     
     
       11. An image processing apparatus comprising: 
       a) input means for inputting successive image data;  
       b) cell generation means for generating a cell serving as a processing unit of area division of the image data and detecting a motion vector in units of generated cells;  
       c) initial division means for initially dividing the image data and initializing the cell of the image data;  
       d) area label determination means for calculating a probability at which the cell has a predetermined area label, in accordance with the motion vector and an area label of a neighboring cell and determining an area label of the cell from a calculation result; and  
       e) area division means for dividing the image data into areas of respective objects on the basis of an output from said area label determination means,  
       wherein said initial division means executes initial division of image data of a processing frame using an area division result of image data of a neighboring frame.  
     
     
       12. An apparatus according to  claim 11 , further comprising scene change detection means for detecting a scene change of an image, and 
       wherein said initial division means executes initial division of image data of the processing frame without using an area division result of image data of a neighboring frame when said scene change detection means detects a scene change of an image.  
     
     
       13. An apparatus according to  claim 11 , wherein said area label determination means calculates the probability using a model of a Markov random field. 
     
     
       14. An apparatus according to  claim 11 , wherein said cell generation means detects an average color vector in units of generated cells. 
     
     
       15. An apparatus according to  claim 14 , wherein said area label determination means calculates the probability in accordance with the average color vector and an area label of a neighboring cell. 
     
     
       16. An apparatus according to  claim 15 , wherein said area label determination means calculates the probability using a model of a Markov random field. 
     
     
       17. An apparatus according to  claim 11 , wherein said cell generation means detects a vector representing a spatial frequency in units of generated cells. 
     
     
       18. An apparatus according to  claim 17 , wherein said area label determination means calculates the probability in accordance with the vector representing the spatial frequency and an area label of a neighboring cell. 
     
     
       19. An apparatus according to  claim 18 , wherein said area label determination means calculates the probability using a model of a Markov random field. 
     
     
       20. An apparatus according to  claim 11 , wherein said cell generation means recursively divides the image data until a predetermined division condition is satisfied, and sets a divided block as a cell. 
     
     
       21. An apparatus according to  claim 11 , wherein said input means includes image sensing means for sensing a subject image and outputting the serial image data. 
     
     
       22. An apparatus according to  claim 11 , further comprising encoding means for encoding each area divided by said area division means. 
     
     
       23. An apparatus according to  claim 22 , wherein said encoding means executes encoding processing complying with an MPEG-4 standard. 
     
     
       24. An apparatus according to  claim 22 , further comprising recording means for recording image data encoded by said encoding means on a recording medium. 
     
     
       25. An apparatus according to  claim 22 , further comprising transmission means for transmitting image data encoded by said encoding means to an external device. 
     
     
       26. An image processing method comprising the steps of: 
       a) inputting successive image data;  
       b) dividing the image data into blocks each formed from a plurality of pixels;  
       c) detecting a motion vector of each block;  
       d) calculating a probability at which the block has a predetermined area label, in accordance with the motion vector and an area label of a neighboring block;  
       e) determining an area label of each block in accordance with a calculation result; and  
       f) dividing the image data into areas of respective objects in accordance with the area label determined in the determination step.  
     
     
       27. An image processing method comprising the steps of: 
       a) inputting successive image data;  
       b) generating a cell serving as a processing unit of area division of the image data and detecting a motion vector in units of generated cells;  
       c) initially dividing the image data and initializing the cell of the image data;  
       d) calculating a probability at which the cell has a predetermined area label, in accordance with the motion vector and an area label of a neighboring cell, and determining an area label of the cell from a calculation result; and  
       e) dividing the image data into areas of respective objects on the basis of the area label determined in the area label determination step,  
       wherein the initial division step comprises executing initial division of a processing frame using an area division result of image data of a neighboring frame.  
     
     
       28. A recording medium storing program codes of the image processing step, the program codes comprising: 
       a) a code of the input step of inputting successive image data;  
       b) a code of the block division step of dividing the image data into blocks each formed from a plurality of pixels;  
       c) a code of the detection step of detecting a motion vector of each block;  
       d) a code of the calculation step of calculating a probability at which the block has a predetermined area label, in accordance with the motion vector and an area label of a neighboring block;  
       e) a code of the determination step of determining an area label of each block in accordance with a calculation result of the calculation step; and  
       f) a code of the area division step of dividing the image data into areas of respective objects in accordance with the area label determined in the determination step.  
     
     
       29. A recording medium storing program codes of the image processing step, the program codes comprising: 
       a) a code of the input step inputting successive image data;  
       b) a code of the cell generation step of generating a cell serving as a processing unit of area division of the image data and detecting a motion vector in units of generated cells;  
       c) a code of the initial division step of initially dividing the image data and initializing the cell of the image data;  
       d) a code of the area label determination step of calculating a probability at which the cell has a predetermined area label, and determining an area label of the cell from a calculation result; and  
       e) a code of the area division step of dividing the image data into areas of respective objects on the basis of the area label determined in the area label determination step,  
       wherein the initial division step comprises executing initial division of image data of a processing frame using an area division result of image data of a neighboring frame.