Image index generation based on similarities of image features

Abstract

Embodiments of the present application relate to an image index generation method, system, a device, and a computer program product. An image index generation method is provided. The method includes selecting an image included in an image library for which an image index is to be generated, determining at least one target region included in the image, extracting visual features from the determined at least one target region, determining a similarity value of the selected image and image included in the image library based on the extracted visual features, determining image categories to which the images belong to based on the determined similarity values among the images, and assigning category identifiers to the images in accordance with an identifier assignment method, the identifier assignment method assigns the same category identifiers to images belonging to the same image category, and different category identifiers to images belonging to different image categories.

What is claimed is: 
     
       1. A method for managing images, comprising:
 selecting, from a storage device, an image included in an image library for which an image index is to be generated; 
 determining at least one target region included in the image, wherein at least a part of a foreground of the image is included in the at least one target region, and a background of the image is not included in the at least one target region; 
 extracting a set of one or more visual features from the determined at least one target region, the set of one or more visual features extracted from the determined at least one target region including a scale-invariant feature transform (SIFT) feature, wherein the extracting of the set of one or more visual features includes:
 for the determined at least one target region in the image:
 performing a SIFT to determine each of key pixel points in the at least one target region; and 
 separately determining an N-dimensional feature vector corresponding to the each of the key pixel points, N being an integer; and 
 
 for the N-dimensional feature vector corresponding to the each of the key pixel points:
 comparing a distance between the N-dimensional feature vector corresponding to a key pixel point and the N-dimensional feature vectors corresponding to the each of the key pixel points in a predetermined sample image library; 
 determining a first key pixel point in the image library, the first key pixel point having a smallest distance from the N-dimensional feature vector corresponding to the each of the key pixel points; 
 determining a number assigned to the first key pixel point; and 
 forming a vector comprising the numbers corresponding to the first key pixel points determined for the each of the key pixel points, wherein the vector serves as the SIFT features extracted from the target region; 
 
 
 determining, using one or more computer processors, a similarity value of the selected image and another image included in the image library based on the extracted set of one or more visual features; 
 determining image categories to which the images included in the image library belong, based at least in part on the determined similarity value of the selected image and the other image; and 
 assigning category identifiers to the images included in the image library, wherein a same category identifier is assigned to images belonging to a same image category, and different category identifiers are assigned to images belonging to different image categories. 
 
     
     
       2. The method as described in  claim 1 , wherein:
 the set of one or more visual features extracted from the determined at least one target region further includes a color feature, a shape feature, a texture feature, or any combination thereof. 
 
     
     
       3. The method as described in  claim 1 , wherein the determining of the similarity value of the selected image and the other image included in the image library based on the set of one or more visual features comprises:
 determining text annotation information corresponding to different images based on descriptive information of the different images included in the image library; 
 determining correlation values used to measure degree of correlation between the text annotation information of the different images; 
 based on the determined correlation values, allocating the images included in the image library to a plurality of image sets using a hierarchical clustering method; and 
 for one of the image sets:
 determining visual feature vectors corresponding to the different images based on the set of one or more visual features; and 
 determining similarity values among the different images based on the determined visual feature vectors corresponding to the different images. 
 
 
     
     
       4. The method as described in  claim 1 , wherein the determining of the similarity value of the selected image and the other image comprises:
 determining visual feature vectors corresponding to different images based on the set of one or more visual features; and 
 determining the similarity values among the different images based on the determined visual feature vectors corresponding to the different images. 
 
     
     
       5. The method as described in  claim 1 , wherein the determining of the at least one target region in the image comprises:
 performing an image smoothing operation on the image; and 
 using each pixel point in the image on which the image smoothing operation has been performed as a seed to perform a region growing operation to segment the image into a plurality of regions, wherein the at least one target region is determined among the plurality of regions. 
 
     
     
       6. The method as described in  claim 1 , wherein the determining of the image categories to which the images included in the image library belong comprises:
 performing a clustering operation on the images included in the image library based on the similarity value of the selected image and the other image, in accordance with a clustering technique, to determine the image categories to which the images included in the image library belong. 
 
     
     
       7. The method as described in  claim 1 , wherein N is 128. 
     
     
       8. An image generation device, comprising:
 at least one processor configured to:
 select an image included in an image library for which an image index is to be generated; 
 determine at least one target region included in the image, wherein at least a part of a foreground of the image is included in the at least one target region, and a background of the image is not included in the at least one target region; 
 extract a set of one or more visual features from the determined at least one target region, the set of one or more visual features extracted from the determined at least one target region including a scale-invariant feature transform (SIFT) feature, wherein the extracting of the set of one or more visual features includes:
 for the determined at least one target region in the image:
 performing a SIFT to determine each of key pixel points in the at least one target region; and 
 separately determining an N-dimensional feature vector corresponding to the each of the key pixel points, N being an integer; and 
 
 for the N-dimensional feature vector corresponding to the each of the key pixel points:
 comparing a distance between the N-dimensional feature vector corresponding to a key pixel point and the N-dimensional feature vectors corresponding to the each of the key pixel points in a predetermined sample image library; 
 determining a first key pixel point in the image library, the first key pixel point having a smallest distance from the N-dimensional feature vector corresponding to the key pixel point; 
 determining a number assigned to the first key pixel point; and 
 forming a vector comprising the numbers corresponding to the first key pixel points determined for the each of the key pixel points, wherein the vector serves as the SIFT features extracted from the target region; 
 
 
 determine a similarity value of the selected image and another image included in the image library based on the extracted set of one or more visual features; 
 determine image categories to which the images included in the image library belong based on the determined similarity value of the selected image and the other image; and 
 assign category identifiers to the images included in the image library, wherein a same category identifier is assigned to images belonging to a same image category, and different category identifiers are assigned to images belonging to different image categories; and 
 
 a memory coupled to the at least one processor and configured to provide the at least one processor with instructions. 
 
     
     
       9. The device as described in  claim 8 , wherein:
 the set of one or more visual features extracted from the determined at least one target region further includes a color feature, a shape feature, a texture feature, or any combination thereof. 
 
     
     
       10. The device as described in  claim 8 , wherein to determine the similarity value of the selected image and the other image included in the image library based on the extracted set of one or more visual features comprises:
 determine text annotation information corresponding to different images based on descriptive information of the different images included in the image library; 
 determine correlation values used to measure degree of correlation between the text annotation information of the different images; 
 based on the determined correlation values, allocate the images included in the image library to a plurality of image sets using a hierarchical clustering method; and 
 for one of the image sets:
 determine visual feature vectors corresponding to the different images based on the set of one or more visual features; and 
 determine similarity values among the different images based on the determined visual feature vectors corresponding to the different images. 
 
 
     
     
       11. The device as described in  claim 8 , wherein to determine the similarity value of the selected image and the other image comprises:
 determine visual feature vectors corresponding to different images based on the set of one or more of visual features; and 
 determine the similarity values among the different images based on the determined visual feature vectors corresponding to the different images. 
 
     
     
       12. The device as described in  claim 8 , wherein to determine the at least one target region in the image comprises:
 perform an image smoothing operation on the image; and 
 use each pixel point in the image on which the image smoothing operation has been performed as a seed to perform a region growing operation to segment the image into a plurality of regions, wherein the at least one target region is determined among the plurality of regions. 
 
     
     
       13. The device as described in  claim 8 , wherein to determine the image categories to which the images included in the image library belong comprises:
 perform a clustering operation on the images included in the image library based on the similarity value of the selected image and the other image, in accordance with a clustering technique, to determine the image categories to which the images included in the image library belong. 
 
     
     
       14. A computer program product for generating an image index, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions for:
 selecting an image included in an image library for which the image index is to be generated; 
 determining at least one target region included in the image, wherein at least a part of a foreground of the image is included in the at least one target region, and a background of the image is not included in the at least one target region; 
 extracting a set of one or more visual features from the determined at least one target region, the set of one or more visual features extracted from the determined at least one target region including a scale-invariant feature transform (SIFT) feature, wherein the extracting of the set of one or more visual features includes:
 for the determined at least one target region in the image:
 performing a SIFT to determine each of key pixel points in the at least one target region; and 
 separately determining an N-dimensional feature vector corresponding to the each of the key pixel points, N being an integer; and 
 
 for the N-dimensional feature vector corresponding to the each of the key pixel points:
 comparing a distance between the N-dimensional feature vector corresponding to a key pixel point and the N-dimensional feature vectors corresponding to the each of the key pixel points in a predetermined sample image library; 
 determining a first key pixel point in the image library, the first key pixel point having a smallest distance from the N-dimensional feature vector corresponding to the key pixel point; 
 determining a number assigned to the first key pixel point; and 
 forming a vector comprising the numbers corresponding to the first key pixel points determined for the each key pixel point, wherein the vector serves as the SIFT features extracted from the target region; 
 
 
 determining a similarity value of the selected image and another image included in the image library based on the extracted set of one or more visual features; 
 determining image categories to which the images included in the image library belong, based at least in part on the determined similarity value of the selected image and the other image; and 
 assigning category identifiers to the images included in the image library, wherein a same category identifier is assigned to images belonging to a same image category, and different category identifiers are assigned to images belonging to different image categories. 
 
     
     
       15. The computer program product as described in  claim 14 , wherein:
 the set of one or more visual features extracted from the determined at least one target region further includes a color feature, a shape feature, a texture feature, or any combination thereof. 
 
     
     
       16. The computer program product as described in  claim 14 , wherein the determining of the similarity value of the selected image and the other image included in the image library based on the extracted visual features comprises:
 separately determining text annotation information corresponding to different images based on descriptive information of the different images included in the image library; 
 determining correlation values used to measure a degree of correlation between the text annotation information of the different images; 
 based on the determined correlation values, allocating the images included in the image library to a plurality of image sets using a hierarchical clustering method; and 
 for one of the image sets:
 determining visual feature vectors corresponding to the different images based on the set of one or more visual features; and 
 determining similarity values among the different images based on the determined visual feature vectors corresponding to the different images. 
 
 
     
     
       17. The computer program product as described in  claim 14 , wherein the determining of the similarity value of the selected image and the other image comprises:
 determining visual feature vectors corresponding to different images based on the set of one or more visual features; and 
 determining the similarity values among the different images based on the determined visual feature vectors corresponding to the different images. 
 
     
     
       18. The computer program product as described in  claim 14 , wherein the determining of the at least one target region in the image comprises:
 performing an image smoothing operation on the image; 
 using each pixel point in the image on which the image smoothing operation has been performed as a seed to perform a region growing operation to segment the image into a plurality of regions, wherein the at least one target region is determined among the plurality of regions. 
 
     
     
       19. The computer program product as described in  claim 14 , wherein the determining of the image categories to which the images included in the image library belong comprises:
 performing a clustering operation on the images included in the image library based on the similarity value of the selected image and the other image, in accordance with a clustering technique, to determine the image categories to which the images included in the image library belong. 
 
     
     
       20. The computer program product as described in  claim 14 , wherein N is 128. 
     
     
       21. A method for managing images, comprising:
 determining at least one target region included in an image included in an image library to be searched, wherein at least a part of a foreground of the image is included in the at least one target region, and a background of the image is not included in the at least one target region; 
 extracting a set of one or more visual features from the determined at least one target region; 
 performing, based at least in part on the set of one or more visual features, a clustering operation on images included in the image library, wherein the performing of the clustering operation on the images comprises:
 obtaining descriptive information respectively associated with the images; 
 obtaining an image set by clustering the images based at least in part on descriptive information respectively associated with the images; and 
 clustering at least images included in the image set based at least in part on at least one of the set of one or more visual features respectively extracted from the images included in the image set; 
 
 assigning category identifiers to the clustered images, wherein a same category identifier is assigned to a cluster of images belonging to a same image category, and different category identifiers are assigned to a cluster of images belonging to different image categories; 
 determining, using one or more processors, at least one image in the image library corresponding to a category identifier which matches a category identifier of the image to be searched; 
 determining, using the one or more processors, a similarity value for the determined at least one image and the image to be searched; and 
 selecting, using the one or more processors, an image from the at least one image based on the similarity value. 
 
     
     
       22. A computer program product for generating an image index, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions for:
 determining at least one target region included in an image included in an image library to be searched, wherein at least a part of a foreground of the image is included in the at least one target region, and a background of the image is not included in the at least one target region; 
 extracting a set of one or more visual features from the determined at least one target region, the set of one or more visual features extracted from the determined at least one target region including a scale-invariant feature transform (SIFT) feature, wherein the extracting of the set of one or more visual features includes:
 for the determined at least one target region in the image:
 performing a SIFT to determine each of key pixel points in the at least one target region; and 
 separately determining an N-dimensional feature vector corresponding to the each of the key pixel points, N being an integer; and 
 
 for the N-dimensional feature vector corresponding to the each of the key pixel points:
 comparing a distance between the N-dimensional feature vector corresponding to a key pixel point and the N-dimensional feature vectors corresponding to the each of the key pixel points in a predetermined sample image library; 
 determining a first key pixel point in the image library, the first key pixel point having a smallest distance from the N-dimensional feature vector corresponding to the key pixel point; 
 determining a number assigned to the first key pixel point; and 
 forming a vector comprising the numbers corresponding to the first key pixel points determined for the each key pixel point, wherein the vector serves as the SIFT features extracted from the target region; 
 
 
 performing, based at least in part on the set of one or more visual features, a clustering operation on images included in the image library; 
 assigning category identifiers to the clustered images, wherein a same category identifier is assigned to a cluster of images belonging to a same image category, and different category identifiers are assigned to a cluster of images belonging to different image categories; 
 determining at least one image in the image library corresponding to a category identifier which matches a category identifier of the image to be searched; 
 determining a similarity value for the determined at least one image and the image to be searched; and 
 selecting an image from the at least one image based on the similarity value.