Multi-stage image matching techniques

Abstract

In some implementations, an image recognition system is capable of using an iterative multi-stage image matching procedure to improve the accuracy and/or reliability of matching a search image with a small number of distinctive feature points to reference images. In the first stage, the system compares a search template for the search image to reference templates of each image within the set of reference images. The system determines a best-matched reference image from among the reference images of the set of reference images. In the second stage, the system compares the hybrid search template to one or more reference templates of reference images that is not the best-matched reference image. The comparison can be used to either compute match scores for reference images corresponding to the one or more reference templates.

What is claimed is: 
     
         1 . A method comprising:
 obtaining a search template of a search image generated during a first stage matching operation between the search image and a reference image;   obtaining data indicating a set of reference templates of a set of reference images, wherein the set of reference images does not include the reference image;   computing, during a second stage matching operation, scores for the set of reference templates, wherein each match score indicates a respective similarity between the search template and a corresponding reference template included in the set of reference images; and   providing the match scores as output of the second stage matching operation.   
     
     
         2 . The method of  claim 1 , wherein:
 the search template comprises:
 a set of search feature points within the search image, and 
 one or more search descriptors for each search feature point included in the set of search feature points; and 
   each reference template included in the set of reference templates comprises:
 a set of reference feature points within the reference image, and 
 one or more reference descriptors for each reference feature point included in the set of reference feature points. 
   
     
     
         3 . The method of  claim 2 , further comprising:
 comparing the set of search feature points and the set of reference feature points based on search descriptors specified by the search template and reference descriptors specified by a reference template; and   determining that the reference image is a best-matched image of the search image based on comparing the set of search feature points and the set of reference feature points. ATT′Y DOCKET No.:  42049 - 0044002     
     
     
         4 . The method of  claim 3 , wherein comparing the set of search feature points and the set of reference feature points comprises determining a number of reference features points, within the set of reference feature points, that have a reference descriptor matching a search descriptor of a mated search feature point. 
     
     
         5 . The method of  claim 2 , wherein the search template is generated during the first stage matching operation based on combining the set of search feature points and the set of reference feature points into a new template for the search image. 
     
     
         6 . The method of  claim 2 , further comprising, during the first stage matching operation:
 comparing the set of search feature points and the set of reference feature points based on search descriptors specified by the search template and reference descriptors specified by a reference template;   determining respective mated regions within each of the search image and the reference image based on comparing the set of search feature points and the set of reference feature points; and   generating the search template by combining search feature points included in a first mated region of the search image and reference feature points included in a second mated region of the reference image.   
     
     
         7 . The method of  claim 2 , wherein the set of search feature points comprises one or more of a ridge intersection point, a ridge edge point, or a corner point. 
     
     
         8 . A system comprising:
 one or more computing devices; and   one or more storage devices storing instructions that, when executed by the one or more computing devices, cause the one or more computing devices to perform operations comprising:   obtaining a search template of a search image generated during a first stage matching operation between the search image and a reference image;   ATT′Y DOCKET No.:  42049 - 0044002  obtaining data indicating a set of reference templates of a set of reference images, wherein the set of reference images does not include the reference image;   computing, during a second stage matching operation, scores for the set of reference templates, wherein each match score indicates a respective similarity between the search template and a corresponding reference template included in the set of reference images; and   providing the match scores as output of the second stage matching operation.   
     
     
         9 . The system of  claim 8 , wherein:
 the search template comprises:   a set of search feature points within the search image, and one or more search descriptors for each search feature point included in the set of search feature points; and   each reference template included in the set of reference templates comprises:   a set of reference feature points within the reference image, and one or more reference descriptors for each reference feature point included in the set of reference feature points.   
     
     
         10 . The system of  claim 9 , wherein the operations further comprise:
 comparing the set of search feature points and the set of reference feature points based on search descriptors specified by the search template and reference descriptors specified by a reference template; and   determining that the reference image is a best-matched image of the search image based on comparing the set of search feature points and the set of reference feature points.   
     
     
         11 . The system of  claim 10 , wherein comparing the set of search feature points and the set of reference feature points comprises determining a number of reference ATT′Y DOCKET No.:  42049 - 0044002  features points, within the set of reference feature points, that have a reference descriptor matching a search descriptor of a mated search feature point. 
     
     
         12 . The system of  claim 9 , wherein the search template is generated during the first stage matching operation based on combining the set of search feature points and the set of reference feature points into a new template for the search image. 
     
     
         13 . The system of  claim 9 , wherein the operations further comprise, during the first stage matching operation:
 comparing the set of search feature points and the set of reference feature points based on search descriptors specified by the search template and reference descriptors specified by a reference template;   determining respective mated regions within each of the search image and the reference image based on comparing the set of search feature points and the set of reference feature points; and   generating the search template by combining search feature points included in a first mated region of the search image and reference feature points included in a second mated region of the reference image.   
     
     
         14 . The system of  claim 9 , wherein the set of search feature points comprises one or more of a ridge intersection point, a ridge edge point, or a corner point. 
     
     
         15 . At least one non-transitory computer-readable storage device storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
 obtaining a search template of a search image generated during a first stage matching operation between the search image and a reference image;   obtaining data indicating a set of reference templates of a set of reference images, wherein the set of reference images does not include the reference image;   computing, during a second stage matching operation, scores for the set of reference templates, wherein each match score indicates a respective similarity ATT′Y DOCKET No.:  42049 - 0044002  between the search template and a corresponding reference template included in the set of reference images; and   providing the match scores as output of the second stage matching operation.   
     
     
         16 . The at least one non-transitory computer-readable storage device of  claim 15 , wherein:
 the search template comprises:   a set of search feature points within the search image, and one or more search descriptors for each search feature point included in the set of search feature points; and   each reference template included in the set of reference templates comprises:   a set of reference feature points within the reference image, and one or more reference descriptors for each reference feature point included in the set of reference feature points.   
     
     
         17 . The at least one non-transitory computer-readable storage device of  claim 16 , wherein the operations further comprise:
 comparing the set of search feature points and the set of reference feature points based on search descriptors specified by the search template and reference descriptors specified by a reference template; and   determining that the reference image is a best-matched image of the search image based on comparing the set of search feature points and the set of reference feature points.   
     
     
         18 . The at least one non-transitory computer-readable storage device of  claim 17 , wherein comparing the set of search feature points and the set of reference feature points comprises determining a number of reference features points, within the set of reference feature points, that have a reference descriptor matching a search descriptor of a mated search feature point. ATT′Y DOCKET No.:  42049 - 0044002   
     
     
         19 . The at least one non-transitory computer-readable storage device of  claim 16 , wherein the search template is generated during the first stage matching operation based on combining the set of search feature points and the set of reference feature points into a new template for the search image. 
     
     
         20 . The at least one non-transitory computer-readable storage device of  claim 16 , wherein the operations further comprise, during the first stage matching operation:
 comparing the set of search feature points and the set of reference feature points based on search descriptors specified by the search template and reference descriptors specified by a reference template;   determining respective mated regions within each of the search image and the reference image based on comparing the set of search feature points and the set of reference feature points; and   generating the search template by combining search feature points included in a first mated region of the search image and reference feature points included in a second mated region of the reference image.