System and method for providing intelligent assistance for food preparation

Abstract

The method and system disclosed herein present a cooking tutorial program that provides intelligent assistance to users during meal preparation, the method and system comprise: receiving a user selection of a recipe; playing a first content item corresponding to a first preparation step of the selected recipe in a first media format; monitoring user's actions performing the first preparation step of the selected recipe; and if the user's actions performing the first preparation step meet first criteria, adjusting the playing of the first content item, including displaying the first content item in a second media format, wherein the second media format differ from the first media format in at least a size and resolution of a portion of the first content item.

What is claimed is: 
     
       1. A method, comprising:
 at a device having one or more processors and memory:
 receiving a user selection of a recipe from a plurality of recipes included in an interactive recipe program, wherein the recipe includes a sequence of ordered preparation steps and each preparation step corresponds to a respective sets of content items; 
 in response to receiving the user selection of the recipe, playing a first content item corresponding to a first preparation step of the selected recipe on a display, wherein the first content item is being played in a first media format; 
 while playing the first content item:
 monitoring user's actions performing the first preparation step of the selected recipe; and 
 controlling one or more cooking appliances based on a first set of control instructions associated with the first preparation step; 
 
 in accordance with a determination that the user's actions performing the first preparation step meets a first criteria, adjusting the playing of the first content item, including displaying the first content item in a second media format, wherein the second media format differs from the first media format in at least a size and resolution of a portion of the first content item, 
 while displaying the first content item, in accordance with a determination that the user's actions are indicative of a question on the first preparation step:
 searching a database associated with the first preparation step to retrieve one or more corresponding answers; and 
 displaying the retrieved one or more answers on the display concurrently with the first content item, and 
 
 while playing the first content item, determining that one or more conditions for playing the first content item in a third format are satisfied; and 
 in response to determining that the one or more conditions for playing the first content item in the third format are satisfied, playing the first content in the third format, wherein the first format is a video format, and the third format is an audio-only format, and wherein the one or more conditions include determining that the user is not looking at the display. 
 
 
     
     
       2. The method of  claim 1 , wherein adjusting the playing of the first content item includes pausing, replaying, or fast-forwarding in the first content item. 
     
     
       3. The method of  claim 1 , including:
 while playing the first content item, detecting, using one or more user action monitoring devices, a current action performed by the user; 
 comparing the detected current action performed by the user with an action threshold of the preparation step; 
 providing, on the display, a warning to the user if the comparison determines that the current action does not fall within the action threshold; and 
 suggesting, on the display, to the user a course of correction on the display. 
 
     
     
       4. A device, comprising:
 one or more processors; and 
 memory storing instructions, the instructions, when executed by the one or more processors, cause the processors to perform operations comprising:
 receiving a user selection of a recipe from a plurality of recipes included in an interactive recipe program, wherein the recipe includes a sequence of ordered preparation steps and each preparation step corresponds to a respective set of content items; 
 in response to receiving the user selection of the recipe, playing a first content item corresponding to a first preparation step of the selected recipe on a display, wherein the first content item is being played in a first media format; 
 while playing the first content item:
 monitoring the user's actions performing the first preparation step of the selected recipe; and 
 controlling one or more cooking appliances based on a first set of control instructions associated with the first preparation step; 
 
 in accordance with a determination that the user's actions performing the first preparation step meets a first criteria, adjusting the playing of the first content item, including displaying the first content item in a second media format, wherein the second media format differs from the first media format in at least a size and resolution of a portion of the first content item, and 
 
 while displaying the first content item, in accordance with a determination that the user's actions are indicative of a question on the first preparation step:
 searching a database associated with the first preparation step to retrieve one or more corresponding answers; 
 displaying the retrieved one or more answers on the display concurrently with the first content item, and 
 
 while playing the first content item, determining that one or more conditions for playing the first content item in a third format are satisfied; and 
 in response to determining that the one or more conditions for playing the first content item in the third format are satisfied, playing the first content in the third format, wherein the first format is a video format, and the third format is an audio-only format, and wherein the one or more conditions include determining that the user is not looking at the display. 
 
     
     
       5. The device of  claim 4 , wherein adjusting the playing of the first content item includes pausing, replaying, or fast-forwarding in the first content item. 
     
     
       6. The device of  claim 4 , wherein the operations include:
 while playing the first content item, detecting, using one or more user action monitoring devices, a current action performed by the user; 
 comparing the detected current action performed by the user with an action threshold of the preparation step; 
 providing, on the display, a warning to the user if the comparison determines that the current action does not fall within the action threshold; and 
 suggesting, on the display, to the user a course of correction on the display. 
 
     
     
       7. A non-transitory computer-readable storage medium storing instructions, the instructions, when executed by one or more processors, cause the processors to perform operations comprising:
 receiving a user selection of a recipe from a plurality of recipes included in an interactive recipe program, wherein the recipe includes a sequence of ordered preparation steps and each preparation step corresponds to a respective set of content items; 
 in response to receiving the user selection of the recipe, playing a first content item corresponding to a first preparation step of the selected recipe on a display, wherein the first content item is being played in a first media format; 
 while playing the first content item:
 monitoring the user's actions performing the first preparation step of the selected recipe; and 
 controlling one or more cooking appliances based on a first set of control instructions associated with the first preparation step; 
 
 in accordance with a determination that the user's actions performing the first preparation step meets a first criteria, adjusting the playing of the first content item, including displaying the first content item in a second media format, wherein the second media format differs from the first media format in at least a size and resolution of a portion of the first content item, and 
 while displaying the first content item, in accordance with a determination that the user's actions are indicative of a question on the first preparation step:
 searching a database associated with the first preparation step to retrieve one or more corresponding answers; 
 displaying the retrieved one or more answers on the display concurrently with the first content item, and 
 
 while playing the first content item, determining that one or more conditions for playing the first content item in a third format are satisfied; and 
 in response to determining that the one or more conditions for playing the first content item in the third format are satisfied, playing the first content in the third format, wherein the first format is a video format, and the third format is an audio-only format, and wherein the one or more conditions include determining that the user is not looking at the display. 
 
     
     
       8. The non-transitory computer-readable storage medium of  claim 7 , wherein adjusting the playing of the first content item includes pausing, replaying, or fast-forwarding in the first content item.