Monitoring product shortages over time

Abstract

Methods, systems, and computer-readable storage media for processing images captured in a retail store and automatically identifying a product shortage are provided. In one implementation, a system may access information describing a desired placement of products on at least one store shelf. The system may analyze a first image captured at a first time to determine that a first quantity of products exceeds a quantity threshold. The system may analyze a second image captured at a second time later than the first time to determine that a second quantity of products is below the quantity threshold. The system may avoid initiating an unscheduled action to cause restocking until a third time later than the second time. Based on a determination that a third quantity of products at the third time are below the quantity threshold, the system may provide information indicative of a product shortage.

1 . A system for processing images captured in a retail store and automatically identifying a product shortage, the system comprising:
 at least one processor configured to:
 access information describing a desired placement of products of a certain product type on at least one store shelf; 
 receive image data from a plurality of image sensors mounted in the retail store, the image data including a plurality of images captured at different times and depicting the at least one store shelf; 
 analyze, from among the plurality of images, a first image captured at a first time to determine that a first quantity of products of the certain product type at the first time exceeds a quantity threshold associated with the information describing the desired placement of products; 
 analyze, from among the plurality of images, a second image captured at a second time later than the first time to determine that a second quantity of products of the certain product type at the second time is below the quantity threshold associated with the information describing the desired placement of products; 
 based on the determination that the first quantity of products of the certain product type at the first time exceeds the quantity threshold, avoid initiating an unscheduled action to cause restocking the at least one store shelf with products of the certain product type at least until a third time later than the second time; 
 analyze, from among the plurality of images, a third image captured at the third time to determine that a third quantity of products of the certain product type at the third time is also below the quantity threshold associated with the information describing the desired placement of products; and 
 based on the determination that the second quantity of products of the certain product type at the second time and the third quantity of products at the third time are below the quantity threshold, provide information indicative of a product shortage at the at least one store shelf. 
   
     
     
         2 . The system of  claim 1 , wherein the at least one processor is further configured to:
 access information indicating that a scheduled restocking event related to the certain product type is about to take place within a period of time after the third time;   provide the information indicative of a product shortage when the period of time is greater than a time threshold; and   withhold providing the information indicative of a product shortage when the period of time is less than the time threshold when the period of time is less than the time threshold.   
     
     
         3 . The system of  claim 1 , wherein the at least one processor is further configured to:
 access information indicative of a quantity of products of the certain product type available in a storage area of the retail store;   provide the information indicative of a product shortage when there are available at least some products of the certain product type in the storage area of the retail store; and   withhold providing the information indicative of a product shortage when there are no available products of the certain product type available in the storage area of the retail store.   
     
     
         4 . The system of  claim 1 , wherein the at least one processor is further configured to:
 access information indicative of a quantity of products of the certain product type available in a storage area of the retail store;   cause a restocking event related to the product type when there are available products of the certain product type available in the storage area of the retail store; and   cause a change to the desired placement of products when there are no available products of the certain product type in the storage area of the retail store.   
     
     
         5 . The system of  claim 1 , wherein the at least one processor is further configured to:
 access information indicative of pending assignments of one or more store employees;   provide the information indicative of a product shortage when there is no pending assignment for restocking the at least one store shelf; and   withhold providing the information indicative of a product shortage when there is a pending assignment for restocking the at least one store shelf.   
     
     
         6 . The system of  claim 1 , wherein the at least one processor is further configured to:
 determine a number of times within a certain time period that the information indicative of a product shortage at the at least one store shelf was previously provided; and   provide the information indicative of a product shortage when the number of times is smaller than a predetermined number; and   withhold providing the information indicative of a product shortage when the number of times is at least equal to the predefined number.   
     
     
         7 . The system of  claim 1 , wherein the at least one processor is further configured to:
 analyze, from among the plurality of images, an additional image captured at an intermediate time between the second time and the third time to determine that a fourth quantity of products of the certain product type at the intermediate time between the second time and the third time is above the quantity threshold associated with the information describing the desired placement of products; and   based on the determination, withhold providing the information indicative of a product shortage.   
     
     
         8 . The system of  claim 1 , wherein the at least one processor is further configured to:
 determine that an elapsed time between the second time and the third time is less than or greater than a predetermined shortage time duration; and   withhold providing the information indicative of a product shortage based on the determination.   
     
     
         9 . The system of  claim 8 , wherein if the elapsed time between the second time and the third time is less than the predetermined shortage time duration, the at least one processor is further configured to trigger capturing of an additional image to determine a quantity of products of the certain product type at a time after the third time. 
     
     
         10 . The system of  claim 8 , wherein if the elapsed time between the second time and the third time is greater than the predetermined shortage time duration, the at least one processor is further configured to provide a recommendation to place products from a differing product type on the at least one store shelf. 
     
     
         11 . The system of  claim 8 , wherein the predetermined shortage time duration is selected based on the product type. 
     
     
         12 . The system of  claim 8 , wherein the predetermined shortage time duration is selected based on the retail store. 
     
     
         13 . The system of  claim 8 , wherein the predetermined shortage time duration is selected based on a quantity of products of the certain product type available in a storage area of the retail store. 
     
     
         14 . The system of  claim 1 , wherein the certain product type is a fresh product and the at least one processor is further configured to analyze the plurality of images to determine a quality of the products on the at least one store shelf, and to determine if there is a product shortage based on both the quantity and determined quality of the products on the at least one store shelf. 
     
     
         15 . The system of  claim 1 , wherein the at least one processor is further configured to:
 analyze a fourth image captured at a fourth time later than the third time and a fifth image captured at a fifth time later than the fourth time to determine that a restocking event occurred between the fourth time and the fifth time;   estimate a time span until restocking based on at least one of the first time and the second time and based on at least one of the fourth time and the fifth time; and   provide information based on the estimated time span until restocking.   
     
     
         16 . The system of  claim 1 , wherein the at least one processor is further configured to:
 provide the information indicative of a product shortage to an external system configured to aggregate product shortage information from a plurality of retail stores; and   receive from the external system information relating to the aggregation of product shortage information from the plurality of retail stores.   
     
     
         17 . The system of  claim 16 , wherein the information relating to the aggregation of product shortage information includes statistics about shortage durations associated with the certain product type. 
     
     
         18 . A method for processing images captured in a retail store and automatically identifying a product shortage, the method comprising:
 accessing information describing a desired placement of products of a certain product type on at least one store shelf;   receiving image data from a plurality of image sensors mounted in a retail store, the image data including a plurality of images captured at different times and depicting the at least one store shelf;   analyzing, from among the plurality of images, a first image captured at a first time to determine that a first quantity of products of the certain product type at the first time exceeds a quantity threshold associated with the information describing the desired placement of products;   analyzing, from among the plurality of images, a second image captured at a second time later than the first time to determine that a second quantity of products of the certain product type at the second time is below the quantity threshold associated with the information describing the desired placement of products;   based on the determination that the first quantity of products of the certain product type at the first time exceeds the quantity threshold, avoiding initiating an unscheduled action to cause restocking the at least one store shelf with products of the certain product type until a third time later than the second time;   analyzing, from among the plurality of images, a third image captured at the third time to determine that a third quantity of products of the certain product type at the third time is also below the quantity threshold associated with the information describing the desired placement of products; and   based on the determination that the second quantity of products of the certain product type at the second time and the third quantity of products at the third time are below the quantity threshold, providing information indicative of a product shortage at the at least one store shelf.   
     
     
         19 . The method of  claim 18 , wherein if the products of the certain product type are placed in two separate locations in the retail store, the information indicative of a product shortage includes details indicative of a quantity of the products in each location. 
     
     
         20 . A computer program product for processing images captured in a retail store and automatically identifying a product shortage, the computer program product being embodied in a non-transitory computer-readable medium and executable by at least one processor, the computer program product including instructions for causing the at least one processor to execute a method comprising:
 accessing information describing a desired placement of products of a certain product type on at least one store shelf;   receiving image data from a plurality of image sensors mounted in a retail store, the image data including a plurality of images captured at different times and depicting the at least one store shelf;   analyzing, from among the plurality of images, a first image captured at a first time to determine that a first quantity of products of the certain product type at the first time exceeds a quantity threshold associated with the information describing the desired placement of products;   analyzing, from among the plurality of images, a second image captured at a second time later than the first time to determine that a second quantity of products of the certain product type at the second time is below the quantity threshold associated with the information describing the desired placement of products;   based on the determination that the first quantity of products of the certain product type at the first time exceeds the quantity threshold, avoiding initiating an unscheduled action to cause restocking the at least one store shelf with products of the certain product type until a third time later than the second time;   analyzing, from among the plurality of images, a third image captured at the third time to determine that a third quantity of products of the certain product type at the third time is also below the quantity threshold associated with the information describing the desired placement of products; and   based on the determination that the second quantity of products of the certain product type at the second time and the third quantity of products at the third time are below the quantity threshold, providing information indicative of a product shortage at the at least one store shelf.   
     
     
         21 .- 82 . (canceled)