Three dimensional visual programming interface for a network of devices

Abstract

Approaches for generating virtual representations of smart objects in a 3D visual programming interface. The interface displays a 3D virtual environment containing virtual objects that represents a real-world environment containing smart objects. The 3D virtual environment displays virtual objects in a manner that is spatially accurate relative to the physical objects in the real-world environment. For each virtual object representing a physical object, a logic node (port node) is displayed, the port node representing the set of functions associated with the physical object. The interface enables users to create, delete, or modify different types of logic nodes (representing functions) and create, delete, or modify links (representing data connections) between logic nodes within the 3D virtual environment. The authoring of the logic nodes and links produces an executable program. Upon executing the program, data flows between the logic nodes are visually represented as particles moving between the logic nodes.

The invention claimed is: 
     
         1 . A computer-implemented method for programming and viewing a network of physical objects within a 3D visual programming interface, the method comprising:
 displaying a 3D virtual environment that includes a first logic node and a second logic node, wherein the 3D virtual environment represents a real-world environment that includes a first physical object and a second physical object, and wherein the first logic node represents a set of functions associated with the first physical object, and the second logic node represents a set of functions associated with the second physical object;   receiving a first set of inputs for creating a first link between the first logic node and the second logic node; and   based on the first set of inputs, displaying the first link between the first logic node and the second logic node within the 3D virtual environment, wherein the first link represents a first data connection between the first physical object and the second physical object in the real-world environment.   
     
     
         2 . The computer-implemented method of  claim 1 , wherein the first and second logic nodes and the first link comprises a set of linked logic nodes representing a set of connected functions, the set of linked logic nodes comprises an executable program, and further comprising executing the program to implement the set of connected functions within the real-world environment. 
     
     
         3 . The computer-implemented method of  claim 1 , wherein the first logic node represents a sensor function associated with the first physical object. 
     
     
         4 . The computer-implemented method of  claim 1 , wherein the second logic node represents an actuator function associated with the second physical object. 
     
     
         5 . The computer-implemented method of  claim 1 , wherein:
 the first logic node is displayed at a first location within the 3D virtual environment based on a first location of the first physical object within the real-world environment; and   the second logic node is displayed at a second location within the 3D virtual environment based on a second location of the second physical object within the real-world environment.   
     
     
         6 . The computer-implemented method of  claim 1 , further comprising receiving a set of inputs for creating a third logic node between the first logic node and the second logic node, a second link between the third logic node and the first logic node, and a third link between the third logic node and the second logic node. 
     
     
         7 . The computer-implemented method of  claim 6 , wherein the third logic node comprises a trigger node, aggregator node, filter node, or a converter node. 
     
     
         8 . The computer-implemented method of  claim 6 , wherein the first logic node comprises a first type of node, and the third logic node comprises a second type of node, the first logic node having a different visual appearance than the third logic node. 
     
     
         9 . The computer-implemented method of  claim 1 , wherein the first logic node, the second logic node, and the first link comprise an executable program, and further comprising:
 executing the program; and   displaying a visual representation of a data flow between the first and second logic nodes comprising particles moving along the first link.   
     
     
         10 . A non-transitory computer-readable medium storing program instructions that, when executed by a processor, cause the processor to perform the steps of:
 displaying a 3D virtual environment that includes a first logic node and a second logic node, wherein the 3D virtual environment represents a real-world environment that includes a first physical object and a second physical object, and wherein the first logic node represents a set of functions associated with the first physical object, and the second logic node represents a set of functions associated with the second physical object;   receiving a first set of inputs for creating a first link between the first logic node and the second logic node; and   based on the first set of inputs, displaying the first link between the first logic node and the second logic node within the 3D virtual environment, wherein the first link represents a first data connection between the first physical object and the second physical object in the real-world environment.   
     
     
         11 . The non-transitory computer-readable medium of  claim 10 , wherein the first and second logic nodes and the first link comprises a set of linked logic nodes representing a set of connected functions, the set of linked logic nodes comprises an executable program, the steps further comprising:
 executing the program to implement the set of connected functions within the real-world environment.   
     
     
         12 . The non-transitory computer-readable medium of  claim 10 , wherein the first logic node represents a sensor function associated with the first physical object. 
     
     
         13 . The non-transitory computer-readable medium of  claim 10 , wherein the second logic node represents an actuator function associated with the second physical object. 
     
     
         14 . The non-transitory computer-readable medium of  claim 10 , further comprising:
 receiving object metadata for the first physical object describing a location of the first physical object within the real-world environment, wherein the first logic node is displayed at a location within the 3D virtual environment based on the object metadata.   
     
     
         15 . The non-transitory computer-readable medium of  claim 10 , further comprising:
 receiving a set of inputs for creating a third logic node between the first logic node and the second logic node, a second link between the third logic node and the first logic node, and a third link between the third logic node and the second logic node;   in response, displaying the third logic node, the second link between the third logic node and the first logic node, and the third link between the third logic node and the second logic node within the 3D virtual environment, wherein the second link represents a second data connection between the third logic node and the first logic node and the third link represents a third data connection between the third logic node and the second logic node, wherein the first link comprises the second link and the third link.   
     
     
         16 . The non-transitory computer-readable medium of  claim 15 , wherein the first logic node and the third logic node are displayed with different visual appearances based on the type of logic node. 
     
     
         17 . The non-transitory computer-readable medium of  claim 10 , wherein the first and second logic nodes and the first link comprises an executable program, the steps further comprising:
 executing the program; and   displaying a visual representation of a data flow between the first and second logic nodes comprising particles moving along the first link.   
     
     
         18 . The non-transitory computer-readable medium of  claim 10 , wherein the visual appearance of particles are based on attributes of the data flow. 
     
     
         19 . A system for programming and viewing a network of physical objects within a 3D visual programming interface, comprising:
 a memory that includes an interface engine; and   a processor that is coupled to the memory and, when executing the interface engine, is configured to perform the steps of:   displaying a 3D virtual environment that includes a first logic node and a second logic node, wherein the 3D virtual environment represents a real-world environment that includes a first physical object and a second physical object, and wherein the first logic node represents a set of functions associated with the first physical object, and the second logic node represents a set of functions associated with the second physical object;   receiving a first set of inputs for creating a first link between the first logic node and the second logic node; and   based on the first set of inputs, displaying the first link between the first logic node and the second logic node within the 3D virtual environment, wherein the first link represents a first data connection between the first physical object and the second physical object in the real-world environment.   
     
     
         20 . The system of  claim 19 , wherein the first and second logic nodes and the first link comprises a set of linked logic nodes representing a set of connected functions, the set of linked logic nodes comprises an executable program, and further comprising executing the program to implement the set of connected functions within the real-world environment.