Method for automatically generating waypoints for imaging shelves within a store

Abstract

One variation of a method for automatically generating waypoints for imaging shelves within a store includes: dispatching a robotic system to autonomously generating a map of a floor space within the store; accessing an architectural metaspace defining target locations and addresses of the set of shelving structures within the store; distorting the architectural metaspace into alignment with the map to generate a normalized metaspace representing real locations and addresses of the set of shelving structures in the store; defining a set of waypoints distributed longitudinally along and offset laterally from a first shelving structure represented in the normalized metaspace based on a known position of an optical sensor in the robotic system; and dispatching the robotic system to record optical data while occupying the set of waypoint optv-mo3 in:#finance s during an imaging routine.

I claim: 
     
         1 . A method for automatically tracking inventory within a store comprising:
 accessing a set of images captured by a robotic system while navigating through the store during a first scan cycle;   for a first image in the set of images:
 detecting a first set of features in the first image; 
 detecting a first optical identifier, applied to a first shelving structure located at a first location in the store, in the first image; 
 based on the first optical identifier, identifying a first product assigned to the first shelving structure; and 
 detecting presence of units of the first product on the first shelving structure based on correlation between features in the first set of features detected in the first image and template visual characteristics, stored in a database, representing the first product; 
   for a second image in the set of images:
 detecting a second optical identifier, applied to a second shelving structure, in the second image; 
 based on the second optical identifier:
 identifying the second shelving structure as a temporary mobile inventory structure; and 
 identifying a second product assigned to the second shelving structure; 
 
 detecting a second location of the second shelving structure within the store; and 
   generating a stock record indicating presence of units of the first product at the first location of the first shelving structure and presence of units of the second product in the temporary mobile inventory structure at the second location during the first scan cycle.   
     
     
         2 . The method of  claim 1 :
 further comprising:
 detecting a second orientation of the second shelving structure within the store based on a position of the second optical identifier within the second image; 
 defining a temporary waypoint proximal the second shelving structure based on the second location of the second shelving structure and the second orientation of the second shelving structure; 
 at the robotic system during the first scan cycle:
 navigating to the temporary waypoint; and 
 capturing a third image, in the set of images, while occupying the temporary waypoint; 
 
 detecting a third set of features in the third image; and 
 detecting presence of units of the second product on the second shelving structure based on correlation between features in the third set of features detected in the third image and template visual characteristics, stored in the database, representing the second product; and 
   wherein generating the stock record comprises, in response to detecting presence of units of the second product on the second shelving structure, generating the stock record confirming presence of units of the second product in the temporary mobile inventory structure at the second location during the first scan cycle.   
     
     
         3 . The method of  claim 2 , wherein defining the temporary waypoint proximal the second shelving structure comprises, based on the second location of the second shelving structure and the second orientation of the second shelving structure, calculating a temporary waypoint location and a temporary waypoint orientation of that, when occupied by the robotic system, locates a front face of the second shelving structure
 in a field of view of a camera on a side of the robotic system; and   at a preset target offset distance from the camera.   
     
     
         4 . The method of  claim 2 , further comprising:
 prior to the first scan cycle:
 accessing a map of a floor space within the store generated based on map data collected by the robotic system while navigating through the store during a mapping routine preceding the first scan cycle; 
 accessing an architectural plan defining target locations of a set of shelving structures within the store; 
 distorting features in the architectural plan into alignment with like features in the map to generate a normalized metaspace representing real locations of the set of shelving structures; 
 identifying the set of shelving structures within the normalized metaspace based on target locations of the set of shelving structures defined in the architectural plan and distorted into alignment with like features in the map; and 
 defining a route along the set of shelving structures identified within the normalized metaspace, the route comprising a first route segment along a first shelving structure, in the set of shelving structures, represented in the normalized metaspace; 
   temporarily appending the temporary waypoint to the route; and   dispatching the robotic system to scan shelving structures in the set of shelving structures in the store while traversing the route during a third scan cycle succeeding the second scan cycle.   
     
     
         5 . The method of  claim 1 :
 further comprising:
 detecting a second set of features in the second image; and 
 detecting presence of units of the second product on the second shelving structure based on correlation between features in the second set of features detected in the second image and template visual characteristics, stored in the database, representing the second product; and 
   wherein generating the stock record comprises, in response to detecting presence of units of the second product on the second shelving structure, generating the stock record confirming presence of units of the second product in the temporary mobile inventory structure at the second location during the first scan cycle.   
     
     
         6 . The method of  claim 1 :
 further comprising detecting a second orientation of the second shelving structure within the store; and   wherein generating the stock record comprises generating the stock record further indicating presence of units of the second product in the temporary mobile inventory structure at the second orientation during the first scan cycle.   
     
     
         7 . The method of  claim 6 , wherein detecting the second location of the second shelving structure within the store and detecting the second orientation of the second shelving structure within the store comprise:
 extracting an image position and an image orientation of the second optical identifier from the second image;   accessing a robotic system position and a robotic system orientation of the robotic system within the store during recordation of the store during recordation of the second image; and   calculating the second location and the second orientation of a front face of the second shelving structure within the store based on the image position, the image orientation, the robotic system position, and robotic system orientation.   
     
     
         8 . The method of  claim 7 , further comprising:
 identifying an aisle in the store occupied by the second shelving structure based on the second location of the second shelving structure;   detecting misorientation of the shelving structure within the store based on a difference between the second orientation of the front face of the second shelving structure and the aisle; and   in response to detecting misorientation of the second shelving structure, generating a prompt to reorient the second shelving structure within the aisle.   
     
     
         9 . The method of  claim 1 , further comprising:
 identifying an aisle in the store occupied by the second shelving structure based on the second location of the second shelving structure;   detecting displacement of the shelving structure within the aisle based on the second location of the second shelving structure; and   in response to detecting displacement of the second shelving structure within the aisle, generating a prompt to relocate the second shelving structure within the aisle.   
     
     
         10 . The method of  claim 1 :
 wherein detecting the second optical identifier comprises detecting the second optical identifier comprising a matrix barcode, applied to the second shelving structure, in the second image; and   wherein identifying the second shelving structure as the temporary mobile inventory structure comprises identifying the second shelving structure as the temporary mobile inventory structure based on the matrix barcode.   
     
     
         11 . The method of  claim 1 :
 wherein identifying the second shelving structure as the temporary mobile inventory structure comprises, based on the second optical identifier, identifying the second shelving structure as a freestanding cardboard popup associated with a particular brand; and   wherein identifying the second product assigned to the second shelving structure comprises, based on the second optical identifier, identifying the second product associated with the particular brand and designated for the freestanding cardboard popup.   
     
     
         12 . The method of  claim 1 :
 wherein identifying the second product assigned to the second shelving structure comprises, based on the second optical identifier, retrieving a second list of products assigned to the second shelving structure;   further comprising:
 retrieving a second set of template images from a database of template images, each template image in the second set of template images comprising visual features of a product in the second list of products; and 
 confirming presence of a unit of the second product, in the second list of products, on the second shelving structure in response to correlation between features in the second image and features in a first template image, in the second set of template images, representing the second product; and 
   wherein generating the stock record comprises, in response to confirming presence of the unit of the second product, generating the stock record indicating presence of units of the second product in the temporary mobile inventory structure at the second location during the first scan cycle.   
     
     
         13 . The method of  claim 1 :
 wherein identifying the second product assigned to the second shelving structure comprises, based on the second optical identifier, retrieving a second list of products assigned to the second shelving structure;   further comprising:
 retrieving a second set of template images from a database of template images, each template image in the second set of template images comprising visual features of a product in the second list of products; 
 generating a count of units of the second product, in the second list of products, on the second shelving structure based on correlations between features in the second image and features in the second set of template images representing the second product; 
 detecting incomplete stocking of the second shelving structure in response to deviation between the count of units of the second product and a target quantity of units of the second product assigned to the second shelving structure; and 
 in response to detecting incomplete stocking of the second shelving structure, generating a temporary restocking prompt for the second shelving structure; and 
   wherein generating the stock record comprises generating the stock record indicating presence of units of the first product at the first location of the first shelving structure and the count of units of the second product in the temporary mobile inventory structure at the second location during the first scan cycle.   
     
     
         14 . A method for automatically tracking inventory within a store comprising:
 accessing a set of images captured by a robotic system while navigating through the store during a first scan cycle;   for a first image in the set of images:
 detecting a first set of features in the first image; 
 detecting a first optical identifier, applied to a first shelving structure located at a first location in the store, in the first image; 
 based on the first optical identifier, identifying a first product assigned to the first shelving structure; and 
 detecting presence of units of the first product on the first shelving structure based on correlation between features in the first set of features detected in the first image and template visual characteristics, stored in a database, representing the first product; 
   for a second image in the set of images:
 detecting a second optical identifier, applied to a second shelving structure, in the second image; 
 detecting a second location of the second shelving structure within the store; 
 identifying the second shelving structure as a temporary mobile inventory structure based on the second location of the second shelving structure intersecting an aisle of the store; and 
 based on the second optical identifier, identifying a second product assigned to the second shelving structure; and 
   generating a stock record indicating presence of units of the first product at the first location of the first shelving structure and presence of units of the second product in the temporary mobile inventory structure at the second location during the first scan cycle.   
     
     
         15 . The method of  claim 14 :
 further comprising:
 detecting a second orientation of the second shelving structure within the store based on a position of the second optical identifier within the second image; 
 defining a temporary waypoint proximal the second shelving structure based on the second location of the second shelving structure and the second orientation of the second shelving structure; 
 at the robotic system:
 navigating to the temporary waypoint; and 
 capturing a third image, in the set of images, while occupying the temporary waypoint; 
 
 detecting a third set of features in the third image; and 
 detecting presence of units of the second product on the second shelving structure based on correlation between features in the third set of features detected in the third image and template visual characteristics, stored in the database, representing the second product; and 
   wherein generating the stock record comprises, in response to detecting presence of units of the second product on the second shelving structure, generating the stock record confirming presence of units of the second product in the temporary mobile inventory structure at the second location during the first scan cycle.   
     
     
         16 . The method of  claim 14 :
 further comprising:
 detecting a second set of features in the second image; and 
 confirming presence of units of the second product on the second shelving structure based on correlation between features in the second set of features detected in the second image and template visual characteristics, stored in the database, representing the second product; and 
   wherein generating the stock record comprises, in response to confirming presence of units of the second product on the second shelving structure, generating the stock record indicating presence of units of the second product in the temporary mobile inventory structure at the second location during the first scan cycle.   
     
     
         17 . The method of  claim 14 :
 further comprising detecting a second optical identifier, applied to the second shelving structure, in the second image; and   wherein identifying the second shelving structure as the temporary mobile inventory structure comprises identifying the second shelving structure as the temporary mobile inventory structure further based on the second optical identifier.   
     
     
         18 . A method for automatically mapping shelving structures within a store comprising:
 accessing a set of images captured by a camera system deployed in the store;   for a first image in the set of images:
 detecting a first optical identifier, applied to a first shelving structure in a set of fixed shelving structures present in the store, in the first image; 
 based on the first optical identifier, identifying a first product assigned to the first shelving structure; 
 detecting presence of a unit of the first product on the first shelving structure based on correlation between features in the first set of features detected in the first image and template visual characteristics, stored in a database, representing the first product; and 
 calculating a first stock condition, of the first shelving structure, indicating presence of the first unit of the first product on the first shelving structure; 
   for a second image in the set of images:
 detecting a second optical identifier, applied to a second shelving structure, in the second image; 
 based on the second optical identifier:
 identifying the second shelving structure as a temporary mobile inventory structure based on the second optical identifier; and 
 identifying a second product assigned to the second shelving structure; 
 
 confirming presence of a second unit of the second product on the second shelving structure based on correlation between features detected in the second image and template visual characteristics, stored in the database, representing the second product; and 
 generating a second stock condition, of the second shelving structure, indicating presence of the second unit of the second product on the second shelving structure; and 
   aggregating the first stock condition of the first shelving structure and the second stock condition of the second shelving structure into a stock record of the store.   
     
     
         19 . The method of  claim 18 :
 further comprising dispatching a mobile robotic system, comprising a set of optical sensors defining the camera system, to autonomously navigate through the store and capture the set of images of inventory structures within the store during a scan cycle; and   wherein accessing the set of images comprises accessing the set of images captured by the mobile robotic system during the scan cycle.   
     
     
         20 . The method of  claim 18 :
 further comprising detecting a second orientation of the second shelving structure within the store; and   wherein generating the stock record comprises generating the stock record further indicating presence of units of the second product in the temporary mobile inventory structure at the second orientation during the first scan cycle.