Image-based kitchen tracking system with metric management and kitchen display system (kds) integration

Abstract

The subject matter of this specification can be implemented in, among other things, methods, systems, computer-readable storage medium. A method includes a processing device receiving image data having one or more image frames indicative of a state of a meal preparation area. The method further includes the processing device determining, based on the image data, one of a meal preparation item or a meal preparation action associated with the state of the meal preparation area. The method further includes the processing device determining, based on the image data, a performance metric associated with the meal preparation item or the meal preparation item or the meal preparation action. The method further includes the processing device causing a notification corresponding to the performance metric to be displayed on a graphical user interface (GUI).

What is claimed is: 
     
         1 . (canceled) 
     
     
         2 . A method, comprising:
 obtaining, by a processing device, image data indicative of a state of an ingredient storage area;   determining, by the processing device, a future state of the ingredient storage area associated with a first future time, based on the image data;   determining a corrective action to be performed by a user based on the future state of the ingredient storage area and the first future time; and   providing an alert to a user indicative of the corrective action.   
     
     
         3 . The method of  claim 2 , wherein the ingredient storage area comprises a first container, and wherein the first future time comprises a time of expiration of a first ingredient disposed within the first container, and wherein the corrective action to be performed by a user comprises replacing the first ingredient before the time of expiration. 
     
     
         4 . The method of  claim 3 , further comprising:
 determining a second future time corresponding to a target state of the ingredient storage area, based on the image data, wherein determining the corrective action to be performed is based on the second future time being later than the first future time.   
     
     
         5 . The method of  claim 4 , further comprising determining a rate of consumption of the first ingredient based on the image data, wherein the target state of the ingredient storage area comprises a depletion of the first ingredient. 
     
     
         6 . The method of  claim 2 , further comprising:
 determining a rate of consumption and a stock amount of a first ingredient based on the image data; and   determining, based on the rate of consumption and the stock amount, a time of depletion of the first ingredient, wherein providing an alert to a user comprises alerting a user to replace the first ingredient.   
     
     
         7 . The method of  claim 2 , further comprising:
 determining a time of expiration of a first ingredient disposed in the ingredient storage area based on the image data;   determining a first duration of time associated with preparation steps for replacing the first ingredient; and   determining a second future time for initiating preparation steps for replacing the first ingredient, based on the time of expiration and the first duration of time, wherein providing the alert to a user comprises providing an indication of the second future time to a user.   
     
     
         8 . The method of  claim 7 , wherein determining the first duration of time comprises: 
 determining a first set of resources for performance of the preparation steps; and   determining available resources for performance of the preparation steps based on the image data.   
     
     
         9 . The method of  claim 7 , wherein preparation steps for replacing the first ingredient comprise one or more of:
 preparing one or more secondary ingredients;   cleaning equipment; or   enabling equipment to reach a target operational state.   
     
     
         10 . The method of  claim 2 , wherein providing the alert to a user comprises causing the alert to be displayed on a graphical user interface. 
     
     
         11 . A non-transitory machine-readable storage medium storing instructions which, when executed, cause a processing device to perform operations comprising:
 obtaining image data indicative of a state of an ingredient storage area;   determining a future state of the ingredient storage area associated with a first future time, based on the image data;   determining a corrective action to be performed by a user based on the future state of the ingredient storage area and the first future time; and   providing an alert to a user indicative of the corrective action.   
     
     
         12 . The non-transitory machine-readable storage medium of  claim 11 , wherein the ingredient storage area comprises a first container, and wherein the first future time comprises a time of expiration of a first ingredient disposed within the first container, and wherein the corrective action to be performed by a user comprises replacing the first ingredient before the time of expiration. 
     
     
         13 . The non-transitory machine-readable storage medium of  claim 12 , wherein the operations further comprise determining a second future time corresponding to a target state of the ingredient storage area based on the image data, wherein the target state of the ingredient storage area comprises a depletion of the first ingredient, and wherein determining the corrective action to be performed is based on the second future time being later than the first future time. 
     
     
         14 . The non-transitory machine-readable storage medium of  claim 11 , wherein the operations further comprise:
 determining a time of expiration of a first ingredient disposed in the ingredient storage area based on the image data;   determining a first duration of time associated with preparation steps for replacing the first ingredient;   determining a second future time for initiating preparation steps for replacing the first ingredient, based on the time of expiration and the first duration of time, wherein providing the alert to a user comprises providing an indication of the second future time to a user.   
     
     
         15 . The non-transitory machine-readable storage medium of  claim 14 , wherein determining the first duration of time comprises:
 determining a first set of resources for performance of the preparation steps; and   determining available resources for performance of the preparation steps based on the image data.   
     
     
         16 . The non-transitory machine-readable storage medium of  claim 11 , wherein providing the alert to a user comprises causing the alert to be displayed on a graphical user interface. 
     
     
         17 . A system, comprising memory and a processing device coupled to the memory, wherein the processing device is configured to:
 obtain image data indicative of a state of an ingredient storage area;   determine a future state of the ingredient storage area associated with a first future time, based on the image data;   determine a corrective action to be performed by a user based on the future state of the ingredient storage area and the first future time; and   provide an alert to a user indicative of the corrective action.   
     
     
         18 . The system of  claim 17 , wherein the ingredient storage area comprises a first container, and wherein the first future time comprises a time of expiration of a first ingredient disposed within the first container, and wherein the corrective action to be performed by a user comprises replacing the first ingredient before the time of expiration. 
     
     
         19 . The system of  claim 18 , wherein the processing device is further to determine a second future time corresponding to a target state of the ingredient storage area based on the image data, wherein the target state of the ingredient storage area comprises a depletion of the first ingredient, and wherein determining the corrective action to be performed is based on the second future time being later than the first future time. 
     
     
         20 . The system of  claim 17 , wherein the processing device is further configured to:
 determine a time of expiration of a first ingredient disposed in the ingredient storage area based on the image data;   determine a first duration of time associated with preparation steps for replacing the first ingredient; and   determine a second future time for initiating preparation steps for replacing the first ingredient, based on the time of expiration and the first duration of time, wherein providing the alert to a user comprises providing an indication of the second future time to a user.   
     
     
         21 . The system of  claim 20 , wherein determining the first duration of time comprises:
 determining a first set of resources for performance of the preparation steps; and determining available resources for performance of the preparation steps based on the image data.