Augmented unification of real and object recognized attributes

Abstract

Systems and methods of using augmented reality (AR) with object recognition (OR) in a physical environment to advance a narrative may be provided. For example, the system may access narrative data representing the narrative. The system may, for a first node of the one or more nodes: generate a user interface associated with the narrative for the first node and access a first decision rule specifying a first physical object to be recognized to advance past the first node in the narrative. The system may further perform image recognition on an image of the physical environment, determine that the first physical object is in the physical environment based on the image recognition, transition from the first node in the narrative based on the first decision rule and the determination, and update the user interface to a second node in the narrative based on the transition from the first node.

What is claims is: 
     
         1 . A system of providing augmented reality (AR) with object recognition in a physical environment to advance a narrative, comprising:
 a processor programmed to:   access narrative data representing the narrative, the narrative data comprising one or more nodes, each node of the one or more nodes being associated with a decision rule that specifies one or more physical objects in the physical environment to be recognized through image recognition to advance past the node in the narrative;   for a first node of the one or more nodes:   generate a user interface associated with the narrative for the first node;
 access a first decision rule specifying a first physical object to be recognized to advance past the first node in the narrative; 
   perform, using an image classifier, image recognition on an image of the physical environment;   determine that the first physical object is in the physical environment based on the image recognition;   transition from the first node in the narrative based on the first decision rule and the determination that the first physical object is in the physical environment; and   update the user interface to a second node in the narrative based on the transition from the first node.   
     
     
         2 . The system of  claim 1 , wherein the first decision rule further specifies a second physical object to be recognized to advance past the first node, wherein the processor is further programmed to:
 advance past the first node only if the second physical object is also determined to be in the physical environment based on the image recognition.   
     
     
         3 . The system of  claim 1 , wherein the first decision rule specifies a plurality of physical objects, including the first physical object, to be recognized to advance past the first node, wherein the processor is further programmed to:
 advance to a second node responsive to a determination that a first set of objects among the plurality of objects is in the physical environment; and   advance to a third node responsive to a determination that a second set of objects among the plurality of objects is in the physical environment.   
     
     
         4 . The system of  claim 1 , wherein the first decision rule further specifies an action to be performed with respect to the first physical object to advance past the first node. 
     
     
         5 . The system of  claim 4 , wherein the user interface displays a first virtual object, the action to be performed comprises causing the first physical object to be within a sufficient proximity to a virtual object displayed by the user interface, and wherein the processor is further programmed to:
 determine whether the first physical object imaged in the user interface is within the sufficient proximity to the virtual object in the user interface.   
     
     
         6 . The system of  claim 5 , wherein to determine whether the first physical object imaged in the user interface is within the sufficient proximity to the virtual object in the user interface, the processor is further programmed to:
 determine whether pixels of the first physical object imaged in the user interface is within a threshold number of pixels of the AR object displayed in the user interface.   
     
     
         7 . The system of  claim 5 , wherein the processor is further programmed to: determine that the first physical object imaged in the user interface is within the sufficient proximity to the virtual object in the user interface;
 determine that an interaction has occurred between the first physical object imaged in the user interface and the virtual object in the user interface based on the determination that the that the first physical object imaged in the user interface is within the sufficient proximity to the virtual object in the user interface; and   display a result of the interaction in the user interface.   
     
     
         8 . The system of  claim 1 , wherein the first decision rule further specifies a user-changed perspective in the physical environment to occur such that the first physical object has a certain size relative to the first physical object. 
     
     
         9 . The system of  claim 1 , wherein the first decision rule further specifies an AR-guided interaction to be performed on the first physical object. 
     
     
         10 . The system of  claim 1 , wherein the processor is further programmed to: generate a virtual avatar to guide the AR-guided interaction to be performed on the first physical object. 
     
     
         11 . A method of providing augmented reality (AR) with object recognition in a physical environment to advance a narrative, the comprising:
 accessing, by a processor, narrative data representing the narrative, the narrative data comprising one or more nodes, each node of the one or more nodes being associated with a decision rule that specifies one or more physical objects in the physical environment to be recognized through image recognition to advance past the node in the narrative;   for a first node of the one or more nodes:   generating, by the processor, a user interface associated with the narrative for the first node;   accessing, by the processor, a first decision rule specifying a first physical object to be recognized to advance past the first node in the narrative;   performing, by the processor, using an image classifier, image recognition on an image of the physical environment;   determining, by the processor, that the first physical object is in the physical environment based on the image recognition;   transitioning, by the processor, from the first node in the narrative based on the first decision rule and the determination that the first physical object is in the physical environment; and   updating, by the processor, the user interface to a second node in the narrative based on the transition from the first node.   
     
     
         12 . The method of  claim 11 , wherein the first decision rule further specifies a second physical object to be recognized to advance past the first node, the method further comprising:
 advancing past the first node only if the second physical object is also determined to be in the physical environment based on the image recognition.   
     
     
         13 . The method of  claim 11 , wherein the first decision rule specifies a plurality of physical objects, including the first physical object, to be recognized to advance past the first node, the method further comprising:
 advancing to a second node responsive to a determination that a first set of objects among the plurality of objects is in the physical environment; and   advancing to a third node responsive to a determination that a second set of objects among the plurality of objects is in the physical environment.   
     
     
         14 . The method of  claim 11 , wherein the first decision rule further specifies an action to be performed with respect to the first physical object to advance past the first node. 
     
     
         15 . The method of  claim 14 , wherein the user interface displays a first virtual object, the action to be performed comprises causing the first physical object to be within a sufficient proximity to a virtual object displayed by the user interface, the method further comprising:
 determining whether the first physical object imaged in the user interface is within the sufficient proximity to the virtual object in the user interface.   
     
     
         16 . The method of  claim 15 , wherein to determining whether the first physical object imaged in the user interface is within the sufficient proximity to the virtual object in the user interface comprises:
 determining whether pixels of the first physical object imaged in the user interface is within a threshold number of pixels of the AR object displayed in the user interface.   
     
     
         17 . The method of  claim 15 , the method further comprising:
 determining that the first physical object imaged in the user interface is within the sufficient proximity to the virtual object in the user interface;   determining that an interaction has occurred between the first physical object imaged in the user interface and the virtual object in the user interface based on the determination that the that the first physical object imaged in the user interface is within the sufficient proximity to the virtual object in the user interface; and   displaying a result of the interaction in the user interface.   
     
     
         18 . The method of  claim 11 , wherein the first decision rule further specifies a user-changed perspective in the physical environment to occur such that the first physical object has a certain size relative to the first physical object. 
     
     
         19 . The method of  claim 11 , wherein the first decision rule further specifies an AR-guided interaction to be performed on the first physical object. 
     
     
         20 . The method of  claim 11 , the method further comprising:
 generating a virtual avatar to guide the AR-guided interaction to be performed on the first physical object.