Augmented reality-enhanced food preparation system and related methods

Abstract

A food preparation system is configured to enhance the efficiency of food preparation operations in a commercial kitchen by displaying instructions on a surface in the kitchen work area. The food preparation system includes a plurality of cameras aimed at a kitchen workspace for preparing the plurality of food items and a processor operable to compute an instruction for a kitchen worker to perform a food preparation step based on one or more types of information selected from order information, recipe information, kitchen equipment information, data from the cameras, and food item inventory information. A projector in communication with the processor visually projects the instruction onto a location in the kitchen workspace for the kitchen worker to observe. Related methods for projecting food preparation instructions are described.

1 . A food preparation system for preparing a plurality of food items in a commercial kitchen environment, the system comprising:
 at least one camera aimed at a kitchen workspace for preparing the plurality of food items;   a processor operable to compute an instruction for a kitchen worker to perform a food preparation step based on data from the at least one camera, order information, and recipe information; and   a projector in communication with the processor and operable to visually project the instruction onto a location in the kitchen workspace for the kitchen worker to observe.   
     
     
         2 . The food preparation system of  claim 1 , wherein the projector is configured to project the instruction on the food item or in close proximity to the food item. 
     
     
         3 . The food preparation system of  claim 1 , wherein the projector comprises a laser for projecting the instruction. 
     
     
         4 . The food preparation system of  claim 1 , wherein the projector comprises AR glasses. 
     
     
         5 . (canceled) 
     
     
         6 . The food preparation system of  claim 1 , wherein the instruction comprises at least one of a text, indicia, symbols, figures, or diagrams. 
     
     
         7 . (canceled) 
     
     
         8 . (canceled) 
     
     
         9 . The food preparation system of  claim 1 , further comprising at least one sensor in addition to the at least one camera. 
     
     
         10 . (canceled) 
     
     
         11 . The food preparation system of  claim 1 , further comprising a communication interface to connect with the internet. 
     
     
         12 . The food preparation system of  claim 1 , wherein the processor is in communication with the POS and adapted to receive the order information. 
     
     
         13 . The food preparation system of  claim 1 , wherein the processor is operable to recognize and locate the food item based on data from the at least one camera. 
     
     
         14 . (canceled) 
     
     
         15 . (canceled) 
     
     
         16 . The food preparation system of  claim 1 , wherein one of the cameras is an IR camera, and wherein the processor is operable to determine next food step based on image data from the IR camera of the food item. 
     
     
         17 . (canceled) 
     
     
         18 . (canceled) 
     
     
         19 . (canceled) 
     
     
         20 . (canceled) 
     
     
         21 . A method to assist a kitchen worker to prepare a plurality of food items in a kitchen workspace in order to complete a customer order, the method comprising:
 receiving the customer order;   computing an instruction for a kitchen worker to perform a food preparation step on the plurality of food items in the kitchen workspace based on order information, camera data, and recipe information; and   projecting the instruction onto a location in the workspace viewable by the kitchen worker.   
     
     
         22 . The method of  claim 21 , wherein the step of projecting is performed directly onto the location. 
     
     
         23 . (canceled) 
     
     
         24 . (canceled) 
     
     
         25 . (canceled) 
     
     
         26 . The method of  claim 21 , further comprising recognizing and locating the food items based on the camera data. 
     
     
         27 . The method of  claim 26 , wherein the step of recognizing and locating is performed with a trained CNN. 
     
     
         28 . The method of  claim 26 , further comprising monitoring a state of the food items. 
     
     
         29 . The method of  claim 28 , further comprising providing an instruction based on a change in the state. 
     
     
         30 . (canceled) 
     
     
         31 . (canceled) 
     
     
         32 . The method of  claim 21 , further comprising computing the total time to carry out the customer order. 
     
     
         33 . (canceled) The method of  claim 32 , further comprising computing a first set of steps to complete a first customer order including computing a duration for each step, and a time to commence each step. 
     
     
         34 . (canceled) 
     
     
         35 . The method of  claim 21 , wherein the step of projecting includes projecting a symbol onto a grill or the food item. 
     
     
         36 . (canceled) 
     
     
         37 . (canceled) 
     
     
         38 . The method of  claim 21 , further comprising displaying the instruction and wherein the displaying comprises use of an AR display adapted to superimpose instructions onto another location shown in the display. 
     
     
         39 . The method of  claim 38 , wherein the location is on top of a food item. 
     
     
         40 . (canceled) 
     
     
         41 . (canceled) 
     
     
         42 . (canceled) 
     
     
         43 . (canceled) 
     
     
         44 . (canceled) 
     
     
         45 . (canceled) 
     
     
         46 . (canceled) 
     
     
         47 . A method to assist a kitchen worker to prepare a plurality of food items in a kitchen workspace to complete a customer order, the method comprising:
 detecting when a cooking threshold has been reached;   computing an instruction for a kitchen worker to perform a food preparation step on the plurality of food items in the kitchen workspace; and   projecting the instruction onto a location in the workspace viewable by the kitchen worker.   
     
     
         48 . The method of  claim 47  wherein the threshold is temperature. 
     
     
         49 .- 88 . (canceled)