Method for detecting and responding to spills and hazards

Abstract

One variation of a method for detecting and responding to hazards within a store includes: autonomously navigating toward an area of a floor of the store; recording a thermal image of the area; recording a depth map of the area of the floor; detecting a thermal gradient in the thermal image; scanning a region of the depth map, corresponding to the thermal gradient detected in the thermal image, for a height gradient; in response to detecting the thermal gradient in the thermal image and in response to detecting absence of a height gradient in the region of the depth map, predicting presence of a fluid within the area of the floor; and serving a prompt to remove the fluid from the area of the floor of the store to a computing device affiliated with the store.

I claim: 
     
         1 . A method for detecting and responding to hazards within a store comprising:
 accessing a thermal image of an area of a floor of the store captured at a first time;   accessing a color image of the area of the floor captured at approximately the first time;   in response to detecting a thermal gradient in a first region of the thermal image and in response to detecting a color gradient less than a threshold color magnitude in a second region of the color image corresponding to the thermal gradient in the thermal image, detecting a clear fluid within the area of the floor; and   in response to detecting the clear fluid within the area of the floor, generating a prompt to remove a spill from the area of the floor, the prompt identifying the spill as clear liquid.   
     
     
         2 . The method of  claim 1 , wherein detecting the clear fluid within the area of the floor comprises detecting the clear fluid within the area of the floor in response to detecting the thermal gradient in the first region of the thermal image and in response to detecting absence of the color gradient in the second region of the color image. 
     
     
         3 . The method of  claim 1 :
 further comprising accessing a depth map of the area of the floor captured at approximately the first time; and   wherein detecting the clear fluid within the area of the floor comprises detecting the clear fluid within the area of the floor further in response to detecting a height gradient less than a threshold height magnitude in a third region of the depth map corresponding to the thermal disparity in the thermal image.   
     
     
         4 . The method of  claim 1 , further comprising:
 accessing a second thermal image of a second area of the floor of the store captured at a second time;   accessing a second color image of the area of the floor captured at approximately the second time;   in response to detecting a second thermal gradient in a third region of the second thermal image and in response to detecting a second color gradient in a fourth region of the second color image corresponding to the second thermal gradient in the second thermal image, detecting a colored fluid within the second area of the floor; and   in response to detecting the colored fluid within the second area of the floor, generating a second prompt to remove a second spill from the second area of the floor, the second prompt identifying the spill as colored liquid.   
     
     
         5 . The method of  claim 4 , further comprising:
 in response to detecting the clear fluid within the area of the floor, serving the prompt, with a first priority, to a computing device affiliated with an associate of the store; and   in response to detecting the colored fluid within the second area of the floor, serving the second prompt, with a second priority less than the first priority, to the computing device.   
     
     
         6 . The method of  claim 5 , further comprising:
 estimating a first opacity of the clear fluid proportional to a first magnitude of the color gradient;   calculating the first priority for removal of the clear fluid from the area of the floor inversely proportional to the first opacity of the clear fluid;   estimating a second opacity of the colored fluid proportional to a second magnitude of the second color gradient; and   calculating the second priority for removal of the colored fluid from the second area of the floor inversely proportional to the second opacity of the colored fluid.   
     
     
         7 . The method of  claim 1 :
 further comprising:
 accessing a sequence of thermal images of the area of the floor captured after the first time; and 
 characterizing a spatial rate of change of thermal gradients, proximal the area of the floor, detected in the sequence of thermal images; and 
   wherein detecting the clear fluid within the area of the floor comprises detecting the clear fluid within the area of the floor further in response to the spatial rate of change of thermal gradients, detected in the sequence of thermal images, falling below a threshold rate of change.   
     
     
         8 . A method for detecting and responding to hazards within a store comprising:
 accessing a thermal image of an area of a floor of the store captured at a first time;   accessing a depth map of the area of the floor captured at approximately the first time;   accessing a color image of the area of the floor captured at approximately the first time;   in response to detecting a thermal disparity in a first region of the thermal image and in response to detecting a height disparity less than a threshold height magnitude in a second region of the depth map corresponding to the thermal disparity in the thermal image, detecting a fluid within the area of the floor;   in response to detecting a color disparity less than a threshold color magnitude in a third region of the depth map corresponding to the thermal gradient in the thermal image, identifying the fluid as clear liquid; and   generating a prompt to remove the fluid from the area of the floor, the prompt identifying the fluid as clear liquid.   
     
     
         9 . The method of  claim 8 , further comprising:
 dispatching a robotic system to autonomously navigate throughout the store during a scan cycle; and   at a robotic system, while autonomously navigating throughout the store during the scan cycle:
 recording the thermal image of the area of the floor proximal the robotic system at the first time; 
 recording the depth map of the area of the floor at approximately the first time; and 
 recording a color image of the area of the floor at approximately the first time. 
   
     
     
         10 . The method of  claim 9 , further comprising:
 accessing a history of locations of fluid spill events in the store;   based on a first quantity of historical fluid spill events occurring in a first aisle in a set of aisles in the store over a period of time according to the history, scheduling the robotic system to traverse the first aisle at a first frequency during a scan cycle;   based on a second quantity of historical fluid spill events, less than the first quantity, occurring in a second aisle in the set of aisles in the store over the period of time according to the history, scheduling the robotic system to traverse the second aisle at a second frequency less than the first frequency during the scan cycle; and   at the robotic system, during the scan cycle:
 autonomously navigating along the first aisle at the first frequency; and 
 autonomously navigating along the second aisle at the second frequency. 
   
     
     
         11 . The method of  claim 9 , further comprising, at the robotic system:
 detecting a perimeter of the fluid in the area of the floor based on the thermal gradient in the thermal image;   autonomously navigating toward the perimeter of the fluid;   holding proximal the perimeter of the fluid; and   outputting an indicator of presence of the fluid.   
     
     
         12 . The method of  claim 11 :
 wherein holding proximal the perimeter of the fluid comprises holding proximal the perimeter of the fluid to physically block access to the area of the floor; and   wherein outputting the indicator of presence of the fluid comprises, while holding proximal the perimeter of the fluid:
 rendering a notification of presence of the fluid nearby on a display integrated into the robotic system; and 
 outputting an audible alert. 
   
     
     
         13 . The method of  claim 9 , further comprising:
 identifying a first set of aisles, in a set of aisles within the store, assigned at least one product type comprising a liquid by a planogram of the store;   identifying a second set of aisles, in the set of aisles within the store, assigned dry goods and excluding product types containing liquids by the planogram of the store; and   at the robotic system:
 autonomously navigating along the first set of aisles within the store at a first frequency during the scan cycle; and 
 autonomously navigating along the second set of aisles within the store at a second frequency, less than the first frequency, during the scan cycle. 
   
     
     
         14 . The method of  claim 8 , further comprising:
 identifying an aisle of the store proximal the fluid based on a location of the area of the floor;   querying a planogram of the store for a list of product types stocked in the aisle;   in response to the list of product types comprising a packaged oil product, predicting that the fluid comprises an oil;   populating an electronic notification with:
 the prompt to remove the fluid from the area of the floor; 
 the location of the area of the floor in the store in the electronic notification; and 
 a recommendation for dry absorbent to remove the fluid; and 
   transmitting the electronic notification to a computing device affiliated with an associate of the store.   
     
     
         15 . The method of  claim 8 , further comprising:
 accessing a second thermal image of a second area of the floor of the store captured at a second time;   accessing a second depth map of the area of the floor captured at approximately the second time;   in response to detecting a second thermal disparity in a fourth region of the second thermal image and in response to detecting a depth disparity greater than the threshold height magnitude in a fifth region of the second depth map corresponding to the thermal disparity in the thermal image, detecting a three-dimensional object within the second area of the floor; and   in response to detecting the three-dimensional object within the second area of the floor, generating a second prompt to remove the three-dimensional object from the second area of the floor.   
     
     
         16 . The method of  claim 15 , further comprising:
 in response to detecting the clear fluid within the area of the floor, serving the prompt, with a first priority, to a computing device affiliated with an associate of the store; and   in response to detecting the three-dimensional object within the second area of the floor, serving the second prompt, with a second priority less than the first priority, to the computing device.   
     
     
         17 . The method of  claim 8 , further comprising:
 accessing a second depth map image of a second area of the floor of the store captured at a second time;   accessing a second color image of the area of the floor captured at approximately the second time;   in response to detecting a second height disparity less than a threshold height magnitude in a third region of the second height map image and in response to detecting a second color disparity in a fourth region of the second color image corresponding to the second height disparity in the second depth map, detecting a second spill within the second area of the floor;   characterizing a linearity of a perimeter of the second color gradient in the fourth region of the second color image;   identifying the second spill as a flat packaging material in response to the linearity of the perimeter of the second color gradient exceeding a threshold linearity; and   in response to detecting the second spill within the second area of the floor, generating a second prompt to remove the second spill comprising flat packaging material from the second area of the floor.   
     
     
         18 . A method for detecting and responding to hazards within a store comprising:
 accessing a thermal image of an area of a floor of the store captured at a first time;   accessing a color image of the area of the floor captured at approximately the first time;   in response to detecting a thermal gradient in a first region of the thermal image, detecting a spill within the area of the floor;   characterizing a color gradient in a second region of the color image corresponding to the thermal gradient in the thermal image;   estimating an opacity of the spill proportional to a magnitude of the color gradient;   calculating a first priority for removal of the spill from the area of the floor inversely proportional to the opacity of the spill; and   in response to detecting the spill within the area of the floor, generating a prompt to remove the spill from the area of the floor, the prompt specifying the first priority.   
     
     
         19 . The method of  claim 18 :
 further comprising accessing a depth map of the area of the floor captured at approximately the first time;   wherein detecting the spill within the area of the floor comprises detecting a fluid within the area of the floor further in response to detecting a height gradient less than a threshold height magnitude in a third region of the depth map corresponding to the thermal disparity in the thermal image; and   wherein generating the prompt to remove the spill from the area of the floor comprises generating the prompt to remove the spill from the area of the floor, the prompt identifying the spill as liquid.   
     
     
         20 . The method of  claim 18 :
 wherein estimating the opacity of the spill comprises identifying the spill as clear liquid in response to the magnitude of the color gradient falling below a threshold color gradient;   wherein calculating the first priority for removal of the spill from the area of the floor comprises assigning the first priority for removal of the spill from the area of the floor in response to identifying the spill as clear liquid; and   further comprising:
 accessing a second thermal image of a second area of the floor of the store captured at a second time; 
 accessing a second color image of the area of the floor captured at approximately the second time; 
 in response to detecting a second thermal gradient in a third region of the second thermal image, detecting a second spill within the second area of the floor; 
 characterizing a second color gradient in a fourth region of the second color image corresponding to the second thermal gradient in the second thermal image; 
 identifying the second spill as colored liquid in response to a second magnitude of the second color gradient exceeding the threshold color gradient; 
 assigning a second priority, less than the first priority, for removal of the second spill from the second area of the floor in response to identifying the second spill as colored liquid; and 
 in response to detecting the second spill within the second area of the floor, generating a second prompt to remove the second spill from the second area of the floor, the second prompt specifying the second priority.