System and method for location based image analysis

Abstract

A system and method for analyzing an image to identify features associated with a particular location or locations.

What is claimed is: 
     
         1 . A method for location-based image analysis, comprising:
 receiving an image to be analyzed by a computational device, said computational device comprising a memory for storing instructions and a processor for executing said instructions, wherein said instructions comprise instructions for:   extracting features from the received image using an image embedding model to generate image embeddings;   determining a relevance of the received image based on a comparison of the image embeddings with a trained image comparison model, wherein the trained image comparison model is selected from the group consisting of an autoencoder and an anomaly detection model;   comparing the received image to a plurality of images with known locations using an Approximate Nearest Neighbor (ANN) search tree model if the received image is determined to be relevant; and   identifying a location associated with the received image based on the comparison with the plurality of images with known locations.   
     
     
         2 . The method of  claim 1 , wherein said image embedding model is selected from the group consisting of a Contrastive Language-Image Pre-training (CLIP), ResNet, VGG (Visual Geometry Group) model variants, EfficientNet models and ViT. 
     
     
         3 . The method of  claim 2 , wherein said trained image comparison model comprises said anomaly detection model, and wherein said anomaly detection model is selected from the group consisting of an IsolationForest algorithm, One-Class SVM (Support Vector Machine), Local Outlier Factor (LOF); DBSCAN (Density-Based Spatial Clustering of Applications with Noise); and K-Means Clustering. 
     
     
         4 . The method of  claim 3 , wherein said anomaly detection model determines a similarity of said received image to a plurality of comparison images according to a similarity measure, wherein if said similarity is above a first threshold, indicating a positive similarity match, said received image is passed to said ANN search tree model; and if said similarity is not above said first threshold, said received image is not passed to said ANN search tree model. 
     
     
         5 . The method of  claim 4 , wherein said anomaly detection model comprises said IsolationForest algorithm. 
     
     
         6 . The method of  claim 1 , wherein the trained image comparison model comprises an autoencoder, and the relevance of the received image is determined based on a mean squared error (MSE) score; wherein if said MSE score is below a first threshold, indicating a positive similarity match, said received image is passed to said ANN search tree model; and if said similarity is not below said first threshold, said received image is not passed to said ANN search tree model. 
     
     
         7 . The method of  claim 6 , wherein the ANN search tree model utilizes an index created from the image embeddings, and the index is optimized based on selected parameters for index construction. 
     
     
         8 . The method of  claim 7 , wherein said instructions further comprise instructions for:
 applying a second threshold to the comparison output from the ANN search tree model to determine if the received image is associated with a location of interest.   
     
     
         9 . The method of  claim 8 , wherein said second threshold is determined based on a distance output from the ANN search tree model, and the distance represents an angular distance between normalized feature vectors of the received image and the plurality of images with known locations. 
     
     
         10 . A system for location-based image analysis, comprising:
 (a) a user computational device configured to receive an image to be analyzed;   (b) an analysis engine communicatively coupled to the user computational device, the analysis engine configured to:   extract features from the received image using an image embedding model to generate image embeddings;   determine a relevance of the received image based on a comparison of the image embeddings with a trained image comparison model, wherein the trained image comparison model is selected from the group consisting of an autoencoder and an anomaly detection model;   compare the received image to a plurality of images with known locations using an Approximate Nearest Neighbor (ANN) search tree model if the received image is determined to be relevant; and   identify a location associated with the received image based on the comparison with the plurality of images with known locations.   (c) a memory for storing the image embeddings, the instructions for executing the analysis engine, and the trained model; and   (d) a processor configured to execute instructions for executing the analysis engine.   
     
     
         11 . The system of  claim 10 , wherein said image embedding model is selected from the group consisting of a Contrastive Language-Image Pre-training (CLIP), ResNet, VGG (Visual Geometry Group) model variants, EfficientNet models and ViT. 
     
     
         12 . The system of  claim 10 or 11 , wherein said trained image comparison model comprises said anomaly detection model, and wherein said anomaly detection model is selected from the group consisting of an IsolationForest algorithm, One-Class SVM (Support Vector Machine), Local Outlier Factor (LOF); DBSCAN (Density-Based Spatial Clustering of Applications with Noise); and K-Means Clustering. 
     
     
         13 . The system of  claim 12 , wherein said anomaly detection model determines a similarity of said received image to a plurality of comparison images according to a similarity measure, wherein if said similarity is below a first threshold, indicating a positive similarity match, said received image is passed to said ANN search tree model; and if said similarity is not below said first threshold, said received image is not passed to said ANN search tree model. 
     
     
         14 . The system of  claim 12 or 13 , wherein said anomaly detection model comprises said IsolationForest algorithm. 
     
     
         15 . The system of  any of the above claims , wherein the trained image comparison model comprises an autoencoder, and the relevance of the received image is determined based on a mean squared error (MSE) score; wherein if said MSE score is below a first threshold, indicating a positive similarity match, said received image is passed to said ANN search tree model; and if said similarity is not below said first threshold, said received image is not passed to said ANN search tree model. 
     
     
         16 . The system of  any of the above claims , wherein the ANN search tree model utilizes an index created from the image embeddings, and the index is optimized based on selected parameters for index construction. 
     
     
         17 . The system of  any of the above claims , wherein said instructions further comprise instructions for:
 applying a second threshold to the comparison output from the ANN search tree model to determine if the received image is associated with a location of interest.   
     
     
         18 . The system of  claim 17 , wherein said second threshold is determined based on a distance output from the ANN search tree model, and the distance represents an angular distance between normalized feature vectors of the received image and the plurality of images with known locations. 
     
     
         19 . A method for preparing images for location-based image analysis, comprising:
 receiving an original image by a computational device, said computational device comprising a memory for storing instructions and a processor for executing said instructions, wherein said instructions comprise instructions for:   analyzing the original image using computer vision object detection to identify and create bounding boxes for specific objects in the image;   generating a cleaned caption for the original image using a large language model, wherein the cleaned caption excludes references to objects within the identified bounding boxes;   performing an inpainting process on the original image to replace content within the bounding boxes based on the cleaned caption, resulting in a cleaned image; and   providing the cleaned image and the cleaned caption as inputs for training a location identification model.   
     
     
         20 . The method of  claim 19 , wherein the specific objects identified for removal include transient objects selected from the group consisting of people, vehicles, and temporary structures. 
     
     
         21 . The method of  claim 19 , further comprising:
 generating synthetic captions for the cleaned image based on class labels associated with the original image;   processing the synthetic captions through a pretrained text encoder to produce text embeddings;   processing the cleaned image through a pretrained image encoder to produce image embeddings; and   combining the text embeddings and image embeddings to create training data for the location identification model.   
     
     
         22 . The method of  claim 19 , wherein the large language model used for generating the cleaned caption is provided with prompts that include bounding box coordinates for the identified objects and instructions to exclude specific types of objects from the caption. 
     
     
         23 . The method of  claim 19 , wherein the inpainting process utilizes an algorithm selected from the group consisting of StableDiffusion Inpainting, Fuse Fooocus SDXL inpainting, FLUX, and Epicrealism.