Augmented reality overlays using position and orientation to facilitate interactions between electronic devices

Abstract

A system includes a first electronic device and a second electronic device. The first electronic device is to display a first augmented reality overlay on imagery of a local environment captured by the first electronic device, the first augmented reality overlay including a depiction of a virtual object from a first perspective that is based on a position and orientation of the first electronic device using a three-dimensional mapping of the local environment. The second electronic device is to display a second augmented reality overlay on imagery of the local environment captured by the second electronic device, the second augmented reality overlay including a depiction of the virtual object from a perspective based on a position and orientation of the second electronic device using the three-dimensional mapping.

What is claimed is: 
     
         1 . A method comprising:
 determining a position and orientation of a first electronic device relative to a second electronic device based on a three-dimensional mapping of an environment of the first and second electronic devices; and   controlling an interaction between the first and second electronic devices based on the position and orientation.   
     
     
         2 . The method of  claim 1 , wherein controlling an interaction between the first and second electronic devices comprises:
 displaying a first augmented reality overlay at the first electronic device, the first augmented reality overlay displaying a first perspective of a virtual object in the environment based on a position and orientation of the first electronic device; and   displaying a second augmented reality overlay at the second electronic device, the second augmented reality overlay displaying a second perspective of the virtual object based on a position and orientation of the second electronic device.   
     
     
         3 . The method of  claim 2 , wherein controlling an interaction between the first and second electronic devices further comprises:
 receiving, at the first electronic device, user input representing a manipulation of the virtual object; and   modifying a depiction of the virtual object in the second augmented reality overlay at the second electronic device in response to the manipulation of the virtual object at the first electronic device.   
     
     
         4 . The method of  claim 3 , wherein:
 the manipulation of the virtual object comprises an insertion of the virtual object into the first augmented reality overlay at the first electronic device; and   modifying the depiction of the virtual object in the second augmented reality overlay comprises initiating display of the virtual object in the second augmented reality overlay.   
     
     
         5 . The method of  claim 3 , further comprising:
 communicating first data representing the manipulation of the virtual object from the first electronic device to a remote processing system; and   communicating a representation of the first data from the remote processing system to the second electronic device;   wherein modifying the depiction of the virtual object in the second augmented reality overlay comprises modifying the depiction of the virtual object based on the representation of the first data.   
     
     
         6 . The method of  claim 1 , wherein controlling an interaction between the first and second electronic devices comprises:
 receiving user input at the first electronic device, the user input representing initiation of motion of a virtual object displayed in a first augmented reality overlay at the first electronic device; and   in response to the user input:
 identifying the second electronic device based on the motion of the virtual object and the position and orientation of the first electronic device relative to the second electronic device; and 
 controlling the first augmented reality overlay at the first electronic device and a second augmented reality overlay at the second electronic device to depict a transfer of the virtual object from the first electronic device to the second electronic device. 
   
     
     
         7 . The method of  claim 1 , wherein controlling an interaction between the first and second electronic devices comprises:
 receiving user input at the first electronic device, the user input representing a manipulation of a virtual object displayed in an augmented reality overlay at the first electronic device; and   communicating data representative the manipulation of the virtual object to a remote processing device for distribution to the second electronic device.   
     
     
         8 . The method of  claim 1 , wherein controlling an interaction between the first and second electronic devices comprises:
 receiving, at the first electronic device, data representing a manipulation of a virtual object displayed in an augmented reality overlay at the second electronic device; and   displaying, at the first electronic device, a representation of the virtual object in an augmented reality overlay based on the manipulation of the virtual object and based on the position and orientation of the first electronic device relative to the second electronic device.   
     
     
         9 . The method of  claim 1 , wherein determining the position and orientation of the first electronic device relative to the second electronic device comprises:
 determining at least one of an orientation and a position of the first electronic device in the environment based on one of: depth information from a depth sensor; and multiview analysis of imagery from a plurality of imaging sensors.   
     
     
         10 . The method of  claim 9 , wherein determining the position and orientation of the first electronic device relative to the second electronic device further comprises:
 determining at least one of an orientation or a position of the second electronic device in the environment based on one of: depth information from a depth sensor; and multiview analysis of imagery from a plurality of imaging sensors.   
     
     
         11 . A method comprising:
 displaying, at a first time at a first electronic device, a first augmented reality overlay on imagery of a local environment captured by the first electronic device, the first augmented reality overlay including a depiction of a virtual object from a first perspective that is based on a position and orientation of the first electronic device using a three-dimensional mapping of the local environment; and   displaying, at a second time at a second electronic device, a second augmented reality overlay on imagery of the local environment captured by the second electronic device, the second augmented reality overlay including a depiction of the virtual object from a perspective based on a position and orientation of the second electronic device using the three-dimensional mapping.   
     
     
         12 . The method of  claim 11 , wherein the first time and the second time are concurrent. 
     
     
         13 . The method of  claim 12 , further comprising:
 transmitting first data from the first electronic device to a remote processing system, the first data representing a user manipulation of the virtual object at the first electronic device; and   receiving second data from the remote processing system at the second electronic device, the second data representative of the user manipulation of the virtual object; and   wherein the second augmented reality overlay comprises a depiction of the user manipulation of the virtual object from the perspective based on the position and orientation of the second electronic device.   
     
     
         14 . The method of  claim 11 , wherein the second time is offset from the first time. 
     
     
         15 . The method of  claim 14 , further comprising:
 transmitting first data from the first electronic device to a remote processing system at the first time, the first data representing a user manipulation of the virtual object at the first electronic device;   storing the first data at the remote processing system;   subsequently accessing, at the second electronic device, a representation of the first data from the remote processing system at the second time; and   wherein the second augmented reality overlay comprises a depiction of the user manipulation of the virtual object from the perspective based on the position and orientation of the second electronic device.   
     
     
         16 . The method of  claim 11 , further comprising:
 separately determining the three-dimensional mapping of the local environment at each of the first electronic device and the second electronic device using visual telemetry.   
     
     
         17 . The method of  claim 11 , further comprising:
 receiving, at the first electronic device, data representative of the three-dimensional mapping of the local environment; and   determining at least one of the position and the orientation of the first electronic device in the local environment based on the three-dimensional mapping and based on visual telemetry at the first electronic device.   
     
     
         18 . The method of  claim 17 , wherein the visual telemetry uses at least one of: depth sensing using a depth sensor; and multiview analysis using at least two imaging sensors. 
     
     
         19 . An electronic device comprising:
 a position/orientation tracking module to determine a position and orientation of the electronic device within a local environment based on a three-dimensional mapping of the local environment; and   a location-based services module coupled to the position/orientation tracking module, the location-based services module to provide an augmented reality overlay on imagery of the local environment captured by the electronic device, the augmented reality overlay including a depiction of a virtual object from a perspective that is based on the position and orientation of the electronic device.   
     
     
         20 . The electronic device of  claim 19 , wherein the position/orientation tracking module is to determine at least one of the position and the orientation of the electronic device based on visual telemetry information. 
     
     
         21 . The electronic device of  claim 20 , wherein the electronic device comprises:
 a depth sensor coupled to the position/orientation tracking module, wherein the position/orientation tracking module is to determine the visual telemetry information using depth information from the depth sensor.   
     
     
         22 . The electronic device of  claim 20 , wherein the electronic device comprises:
 a plurality of imaging sensors coupled to the position/orientation tracking module, wherein the position/orientation tracking module is to determine the visual telemetry information based on multiview analysis of image information from the plurality of imaging sensors.   
     
     
         23 . The electronic device of  claim 19 , further comprising:
 an interface connected to a remote processing system; and   wherein the location-based services module is to receive data representing a manipulation of the virtual object by a user of another electronic device and to modify the depiction of the virtual object by the augmented reality overlay based on the manipulation of the virtual object.   
     
     
         24 . The electronic device of  claim 19 , further comprising:
 a first interface to couple to a remote processing system;   an interface to receive input representative of a manipulation of the virtual object by a user of the electronic device; and   wherein the location-based services module is to provide to the first interface data representative of the manipulation of the virtual object for transmission to the remote processing system for distribution to at least one other electronic device.   
     
     
         25 . A system comprising:
 a first electronic device to display a first augmented reality overlay on imagery of a local environment captured by the first electronic device, the first augmented reality overlay including a depiction of a virtual object from a first perspective that is based on a position and orientation of the first electronic device using a three-dimensional mapping of the local environment; and   a second electronic device to display a second augmented reality overlay on imagery of the local environment captured by the second electronic device, the second augmented reality overlay including a depiction of the virtual object from a perspective based on a position and orientation of the second electronic device using the three-dimensional mapping.   
     
     
         26 . The system of  claim 25 , further comprising:
 a remote processing system coupled to the first and second electronic devices, the remote processing system to communicate data representative of a manipulation of the virtual object by a user of the first electronic device from the first electronic device to the second electronic device; and   wherein the second electronic device is to modify the depiction of the virtual object in the second augmented reality overlay based on the data representative of the manipulation of the virtual object.   
     
     
         27 . The system of  claim 25 , wherein at least one of the first electronic device and the second electronic device is to determine at least one of a position and an orientation using visual telemetry. 
     
     
         28 . The system of  claim 27 , wherein the visual telemetry uses at least one of: depth sensing using a depth sensor; and multiview analysis using at least two imaging sensors. 
     
     
         29 . The system of  claim 25 , wherein the first and second augmented reality overlays are displayed concurrently. 
     
     
         30 . The system of  claim 25 , wherein the first and second augmented reality overlays are displayed at separate times.