Method and apparatus for addressing obstruction in an interface

Abstract

A user, a manipulator such as a hand, and at least one entity such as a virtual or augmented reality object are in an interface such as a 3D environmental interface. The manipulation distance is the distance between a reference feature of the user and a manipulation feature of the manipulator. The entity distance is the distance between the reference feature and an entity feature of the entity. When the manipulation distance becomes greater than the entity distance, the entity is caused to fade, disappear, move out of the way, shrink, etc. so as to be less of an obstruction to the user's field of view, for example to avoid obstructing more distant entities. Other factors than the manipulation distance and entity distance may be considered in determining whether to reduce the obstructivity of the entity, and exceptions to the obstruction relation may be considered.

We claim: 
     
       1. A method, comprising:
 displaying, by a display, a first augmented reality object within an augmented reality interface to a user; 
 displaying, by the display, a second augmented reality object within the augmented reality interface to the user; 
 defining, by a processing device, a first distance range of space as an area between the user and a first point in space at a threshold distance from one of:
 a geometric center of a body of the user; 
 a geometric center of a head of the user; 
 an eye of the user; 
 a retina of the user; or 
 a midpoint between eyes of the user; 
 
 defining, by the processing device, a second distance range of space as an area between the first point in space and a second point in space at a threshold distance from the first point in space; 
 defining, by the processing device, a third distance range of space as an area between second point in space and a third point in space at a threshold distance from the second point in space, wherein:
 the first distance range is closer to the user than the second distance range and the third distance range; and 
 the second distance range is closer to the user than the third distance range; 
 the first augmented reality object is located within the second distance range; and 
 the second augmented reality object is located within the third distance range; 
 
 receiving, by a sensor, an input from an end-effector of a first body part of the user interacting with the augmented reality interface; 
 determining whether the first augmented reality object and the end-effector are both located in the second distance range or the second augmented reality object and the end-effector are both located in the third distance range, wherein:
 the first augmented reality object and the end-effector being located in the second distance range indicates the user intends to interact with the first augmented reality object; and 
 the second augmented reality object and the end-effector being located in the third distance range indicates the user intends to interact with the second augmented reality object; 
 
 in response to the first augmented reality object and the end-effector being located in the second distance range, continuing to display the first augmented reality object at its original visual obstructivity level and the second augmented reality object at its original visual obstructivity level; and 
 in response to the second augmented reality object and the end-effector being located within the third distance range, reducing, by the processing device, the visual obstructivity level of the first augmented reality object relative to the user to allow the user to interact with the second augmented reality object without the first augmented reality object obstructing the user's interaction with the second augmented reality object. 
 
     
     
       2. The method of  claim 1 , wherein the method further comprises:
 establishing, by the processing device, an exception factor; and 
 if the exception factor is satisfied, displaying, by the display, the first augmented reality object as originally displayed. 
 
     
     
       3. The method of  claim 2 , wherein the exception factor comprises a disposition of a second body part of the user. 
     
     
       4. The method of  claim 3 , wherein the end-effector comprises at least one of:
 a hand of the user; 
 a fingertip of the user; or 
 a tip of a stylus held by the user. 
 
     
     
       5. The method of  claim 2 , wherein the exception factor comprises a user input. 
     
     
       6. The method of  claim 5 , wherein the user input comprises at least one of a hand posture, a hand gesture, an eye posture, an eye gesture, or a voice command. 
     
     
       7. The method of  claim 2 , wherein the exception factor comprises determining whether the first augmented reality object is within a line of sight of the user. 
     
     
       8. The method of  claim 1 , wherein the first body part of the user is defined by the user via the augmented reality interface. 
     
     
       9. The method of  claim 1 , wherein the augmented reality interface is selectable by the user. 
     
     
       10. The method of  claim 1 , wherein threshold distance comprises at least one of a linear distance or an absolute distance. 
     
     
       11. The method of  claim 1 , wherein reducing the visual obstructivity level comprises at least one of:
 decreasing an opacity of the first augmented reality object; 
 decreasing the opacity of the first augmented reality object to substantially zero; 
 de-resolving the first augmented reality object; 
 deleting the first augmented reality object; 
 decreasing a size of the first augmented reality object; or 
 changing a position of the first augmented reality object. 
 
     
     
       12. The method of  claim 1 , wherein reducing the visual obstructivity level comprises angularly displacing the first augmented reality object about an axis substantially coinciding with the end-effector. 
     
     
       13. The method of  claim 1 , wherein reducing the visual obstructivity level comprises at least one of:
 displacing the first augmented reality object at least 60 degrees; 
 displacing the first augmented reality object at least 120 degrees; 
 displacing the first augmented reality object at least by an angle such that the first augmented reality object is substantially outside a field of view of the user; or 
 displacing the first augmented reality object such that the first augmented reality object is substantially outside the field of view of the user. 
 
     
     
       14. The method of  claim 1 , wherein the first augmented reality object comprises at least one of a light emission, a light projection, or a hologram. 
     
     
       15. The method of  claim 1 , comprising:
 identifying, by the processing device, a line of sight of the user; and 
 when the first augmented reality object is in the line of sight of the user displaying the first augmented reality object with an original obstructivity level. 
 
     
     
       16. The method of  claim 1 , comprising:
 determining, by the processing device, a direction of a line of sight of the user; and 
 when the line of sight of the user is not toward the first augmented reality object, displaying the first augmented reality object at an original obstructivity level. 
 
     
     
       17. The method of  claim 1 , comprising displaying the first augmented reality object at an original obstructivity level first when the first augmented reality object is not active. 
     
     
       18. The method of  claim 1 , comprising:
 determining an exception time threshold for the first augmented reality object, wherein the exception time threshold is a period of time in which the first augmented reality object is displayed prior to displaying the first augmented reality object at the reduced obstructivity level; 
 determining a length of time that the first augmented reality object has been displayed by the display; and 
 initiating displaying the first augmented reality object at the reduced visual obstructivity level when the length of time that the first augmented reality object has been displayed exceeds the exception time threshold. 
 
     
     
       19. The method of  claim 1 , wherein the end-effector being located in the first distance range indicates that the user does not intend to interact with the first augmented reality object or the second augmented reality object. 
     
     
       20. The method of  claim 1 , further comprising:
 defining, by the processing device, a fourth distance range of space as an area that extends beyond the third point in space, wherein the fourth distance range being further from the user than the first distance range, the second distance range, and the third distance range; 
 determining whether the end-effector is located in the fourth distance range, wherein the end-effector being located in the fourth distance range indicates the user does not intends to interact with the first augmented reality object or the second augmented reality object; and 
 in response to the end-effector being located in the fourth distance range, continuing to display the first augmented reality object at its original visual obstructivity level and the second augmented reality object at its original visual obstructivity level. 
 
     
     
       21. An apparatus, comprising:
 a display to display an augmented reality object within an augmented reality interface to a user; 
 a sensor to receive an input from an end-effector of a body part of the user interacting with the augmented reality interface; and 
 a processor in communication with the sensor and the display, wherein the processor is to:
 define a vertical plane extending through a center of mass of the user; 
 determine a first linear plane-to-point distance between the vertical plane and the end-effector; 
 determine a second linear plane-to-point distance between the vertical plane and the augmented reality object; 
 determine a relative difference between the first linear plane-to-point distance and the second linear plane-to-point distance; 
 determine a reduced obstructivity level for the augmented reality object; and 
 display, via the display, the augmented reality object at the determined reduced obstructivity level when the relative difference is below a threshold distance. 
 
 
     
     
       22. The apparatus of  claim 21 , wherein the sensor comprises at least one of an imager, a stereo pair of imagers, a depth sensor, a structured light sensor, or a time of flight sensor. 
     
     
       23. The apparatus of  claim 21 , wherein the display comprises at least one of an optical see-through visual display, a virtual see-through display, or a stereo visual display. 
     
     
       24. The apparatus of  claim 21 , wherein at least a portion of the apparatus is disposed on a head mounted display. 
     
     
       25. An apparatus, comprising:
 a display to:
 display a first augmented reality object within an augmented reality interface to a user; and 
 display a second augmented reality object within the augmented reality interface to the user; 
 
 a sensor to receive an input from an end-effector of a first body part of the user interacting with the augmented reality interface; 
 a processing device to:
 define a first distance range of space as an area between the user and a first point in space at a threshold distance from one of:
 a geometric center of a body of the user; 
 a geometric center of a head of the user; 
 an eye of the user; 
 a retina of the user; or 
 a midpoint between eyes of the user; 
 
 define a second distance range of space as an area between the first point in space and a second point in space at a threshold distance from the first point in space; 
 define a third distance range of space as an area between second point in space and a third point in space at a threshold distance from the second point in space, wherein:
 the first distance range is closer to the user than the second distance range and the third distance range; and 
 the second distance range is closer to the user than the third distance range; 
 the first augmented reality object is located within the second distance range; and 
 the second augmented reality object is located within the third distance range; 
 
 determine whether the first augmented reality object and the end-effector are both located in the second distance range or the second augmented reality object and the end-effector are both located in the third distance range, wherein:
 the first augmented reality object and the end-effector being located in the second distance range indicates the user intends to interact with the first augmented reality object; and 
 the second augmented reality object and the end-effector being located in the third distance range indicates the user intends to interact with the second augmented reality object; 
 
 in response to the first augmented reality object and the end-effector being located in the second distance range, continuing to display the first augmented reality object at its original visual obstructivity level and the second augmented reality object at its original visual obstructivity level; and 
 in response to the second augmented reality object and the end-effector being located within the third distance range, reducing the visual obstructivity level of the first augmented reality object relative to the user to allow the user to interact with the second augmented reality object without the first augmented reality object obstructing the user's interaction with the second augmented reality object.