Selecting virtual objects in a three-dimensional space

Abstract

Systems and methods for interacting with virtual objects in a three-dimensional space using a wearable system are disclosed. The wearable system can be programmed to permit user interaction with interactable objects in a field of regard (FOR) of a user. The FOR includes a portion of the environment around the user that is capable of being perceived by the user via the AR system. The system can determine a group of interactable objects in the FOR of the user and determine a pose of the user. The system can update, based on a change in the pose or a field of view (FOV) of the user, a subgroup of the interactable objects that are located in the FOV of the user and receive a selection of a target interactable object from the subgroup of interactable objects. The system can initiate a selection event on the target interactable object.

What is claimed is: 
     
       1. A wearable system for selecting a virtual object located in a three-dimensional (3D) space, the wearable system comprising: a display system configured to present virtual objects in a 3D space; a non-transitory data store configured to store interactable objects in the 3D space; a sensor configured to determine a pose of a user; and a hardware processor programmed to communicate with the display system, the data store, and the sensor to: determine the pose of the user based on data received from the sensor; determine a field of view (FOV) of the user based on the pose of the user, the FOV comprising a portion of an environment of the user that is perceived at a given time by the user; identify a group of interactable objects in the FOV; identify a target interactable object in the FOV based least partly on the pose of the user; initiate a selection event associated with the target interactable object; and in response to initiating a selection event on the target interactable object, present, in a two-dimensional (2D) interface, virtual objects within a threshold range of the target interactable object in the 3D space. 
     
     
       2. The system of  claim 1 , wherein the hardware processor is programmed further to index the group of interactable objects based on a coordinate associated with a distance from a position of the target interactable object. 
     
     
       3. The system of  claim 2 , wherein the coordinate comprises a lateral distance, vertical distance, and depth distance relative to the position of the target interactable object. 
     
     
       4. The system of  claim 2 , wherein the hardware processor is programmed further to determine the position of the target interactable object based on the user's pose. 
     
     
       5. The system of  claim 1 , wherein the group of interactable objects is stored in a data structure including an index associated with each interactable object, each index being determined based on a position of the interactable object in the 3D space. 
     
     
       6. The system of  claim 1 , wherein the 2D interface is interactable via a user input device. 
     
     
       7. The system of  claim 1 , wherein to identify the target interactable object in the FOV, the processor is configured to perform at least one of:
 determine a path of a gaze of the user based on the pose of the user and select an interactable object which intersects the path of the gaze as the target interactable object; or select a leftmost or a rightmost interactable object in the user's FOV as the target interactable object, wherein the leftmost or the rightmost interactable object is selected based on indices associated with the group of interactable objects. 
 
     
     
       8. The system of  claim 1 , wherein the processor is configured to initiate the selection event in response to receiving an input from a user input device or detecting a change of the pose of the user. 
     
     
       9. The system of  claim 1 , wherein the processor is further configured to present a focus indicator associated with the target interactable object. 
     
     
       10. The system of  claim 1 , further comprising a geolocation sensor configured to acquire data associated with a location of the user, and wherein the target interactable object comprises a weather application which is programmed to:
 determine the location of the user based on the data acquired by the geolocation sensor; 
 communicate with a remote computing device to acquire weather data based on the location of the user; 
 generate a virtual element associated with the weather data; and 
 superimpose the virtual element in the 3D space of the user. 
 
     
     
       11. A method for selecting a virtual object located in a three-dimensional (3D) space, the method comprising: under control of a hardware processor programmed to communicate with a display system, a data store, and a sensor configured to determine a pose of a user: determining the pose of the user based on data received from the sensor; determining a field of view (FOV) of the user based on the pose of the user, the FOV comprising a portion of an environment of the user that is perceived at a given time by the user; identifying a group of interactable objects in the FOV; identifying a target interactable object in the FOV based least partly on the pose of the user; initiating a selection event associated with the target interactable object; and in response to initiating a selection event on the target interactable object, presenting, in a two-dimensional (2D) interface, virtual objects within a threshold range of the target interactable object in a 3D space. 
     
     
       12. The method of  claim 11 , further comprising indexing the group of interactable objects based on a coordinate associated with a distance from a position of the target interactable object. 
     
     
       13. The method of  claim 12 , wherein the coordinate comprises a lateral distance, vertical distance, and depth distance relative to the position of the target interactable object. 
     
     
       14. The method of  claim 12 , further comprising determining the position of the target interactable object based on the user's pose. 
     
     
       15. The method of  claim 11 , further comprising indexing each interactable object based on a position of the target interactable object in the 3D space. 
     
     
       16. The method of  claim 11 , further comprising storing the group of interactable objects in a data structure including an index associated with each interactable object, each index being determined based on a position of the interactable object in the 3D space. 
     
     
       17. The method of  claim 11 , further comprising receiving user interaction with the 2D interface via a user input device. 
     
     
       18. The method of  claim 11 , further comprising at least one of: determining a path of a gaze of the user based on the pose of the user and select an interactable object which intersects the path of the gaze as the target interactable object; or selecting a leftmost or a rightmost interactable object in the user's FOV as the target interactable object, wherein the leftmost or the rightmost interactable object is selected based on indices associated with the group of interactable objects. 
     
     
       19. The method of  claim 11 , further comprising initiating the selection event in response to receiving an input from a user input device or detecting a change of the pose of the user. 
     
     
       20. The method of  claim 11 , further comprising:
 using a geolocation sensor, acquiring data associated with a location of the user; 
 determining the location of the user based on the data acquired by the geolocation sensor; 
 communicating with a remote computing device to acquire weather data based on the location of the user; 
 generating a virtual element associated with the weather data; and 
 superimposing the virtual element in the 3D space of the user.