Matching an approximately located query image against a reference image set

Abstract

Aspects of the invention pertain to matching a selected image/photograph against a database of reference images having location information. The image of interest may include some location information itself, such as latitude/longitude coordinates and orientation. However, the location information provided by a user's device may be inaccurate or incomplete. The image of interest is provided to a front end server, which selects one or more cells to match the image against. Each cell may have multiple images and an index. One or more cell match servers compare the image against specific cells based on information provided by the front end server. An index storage server maintains index data for the cells and provides them to the cell match servers. If a match is found, the front end server identifies the correct location and orientation of the received image, and may correct errors in an estimated location of the user device.

1 . An image processing method, comprising:
 receiving an image request from a user device, the image request including an image of interest and location metadata for the image of interest;   analyzing the location metadata to select one or more cells to evaluate against the image of interest, each cell having one or more geolocated images and index data associated therewith;   for each selected cell, comparing the image of interest against the index data of that cell;   identifying any matches from the geolocated images of the selected cells based on the compared index data; and   providing the matches.   
     
     
         2 . The image processing method of  claim 1 , wherein the matches are provided along with a match confidence indicator that identifies a likelihood or accuracy of each match. 
     
     
         3 . The image processing method of  claim 1 , wherein updated location metadata for the image of interest is provided to the user device along with the matches. 
     
     
         4 . The image processing method of  claim 1 , wherein the index data is stored in an index storage server, and the index data for each selected cell is accessed with a key representing that cell's unique ID. 
     
     
         5 . The image processing method of  claim 1 , wherein the index data corresponds to features of the geolocated images. 
     
     
         6 . The image processing method of  claim 5 , wherein the features are selected from the set consisting of corners, edges or lines, brightness information and histogram information. 
     
     
         7 . The image processing method of  claim 5 , wherein the geolocated images are stored in an image database and the index data is stored in a cell database. 
     
     
         8 . The image processing method of  claim 5 , wherein the index data is stored in a k-dimensional tree format. 
     
     
         9 . The image processing method of  claim 1 , wherein each cell has a unique ID derived from geolocation coordinates of that cell. 
     
     
         10 . An image processing apparatus, comprising:
 a front end module configured to receive an image request from a user device, the image request including an image of interest and location metadata for the image of interest, the front end module being further configured to analyze the location metadata to select one or more cells to evaluate against the image of interest, each cell having one or more geolocated images and index data associated therewith; and   a cell match module configured to compare the image of interest against the index data of the selected cells and to identify any matches from the geolocated images of the selected cells based on the compared index data.   
     
     
         11 . The image processing apparatus of  claim 10 , wherein the index data corresponds to features of the geolocated images. 
     
     
         12 . The image processing apparatus of  claim 11 , wherein the features are selected from the set consisting of corners, edges or lines, brightness information and histogram information. 
     
     
         13 . A tangible memory on which computer-readable instructions of a computer program are stored, the instructions, when executed by a processor, cause the processor to perform a method, the method comprising:
 receiving an image request from a user device, the image request including an image of interest and location metadata for the image of interest;   analyzing the location metadata to select one or more cells to evaluate against the image of interest, each cell having one or more geolocated images and index data associated therewith;   for each selected cell, comparing the image of interest against the index data of that cell;   identifying any matches from the geolocated images of the selected cells based on the compared index data; and   providing the matches.   
     
     
         14 . The memory of  claim 13 , wherein the method provides the matches along with a match confidence indicator that identifies a likelihood or accuracy of each match.