Projection Casting in Virtual Environments

Abstract

The present technology relates to artificial reality systems. Such systems provide projections a user can create to specify object interactions. For example, when a user wishes to interact with an object outside her immediate reach, she can use a projection to select, move, or otherwise interact with the distant object. The present technology also includes object selection techniques for identifying and disambiguating between objects, allowing a user to select objects both near and distant from the user. Yet further aspects of the present technology include techniques for interpreting various bimanual (two-handed) gestures for interacting with objects. The present technology further includes a model for differentiating between global and local modes for, e.g., providing different input modalities or interpretations of user gestures.

I/We claim: 
     
         1 . A computing system comprising:
 one or more processors; and   a memory storing instructions that, when executed by the one or more processors, cause the computing system to perform a process comprising:   determining casting direction, from an eye of a user, that is away from the user;   continuously monitoring a fist control point based on a position of a part of a first hand of the user;   continuously monitoring a second control point based on a position of a part of a second hand of the user;   defining a rectangle with opposite corners corresponding to the fist control point and the second control point;   generating a pyramid projection that extends along the casting direction, wherein a tip of the pyramid is at a point corresponding to one of the user's eyes and each of the sides of the pyramid intersect with a different one of the lines that forms the rectangle;   identifying multiple objects that at least partially intersect with at least part of the pyramid projection, either simultaneously or at different times; and   selecting the multiple objects.   
     
     
         2 . The computing system of  claim 1 , wherein the process is performed in response to identifying that the user performed a gesture of:
 bringing a thumb-tip on the user's first hand together with a fingertip on the user's first hand to make a first contact point;   bringing a thumb-tip on the user's second hand together with a fingertip on the user's second hand to make a second contact point;   bringing the first contact point together with the second contact point; and   pulling the first contact point away from the second contact point.   
     
     
         3 . The computing system of  claim 1 , wherein the process further comprises:
 in response to the selecting the multiple objects, causing a menu to be attached to the first hand of the user;   identifying an interaction, by the user's second hand, with an item on the menu; and   in response to the interaction, causing a resize or rotation of at least one of the selected multiple objects.   
     
     
         4 . The computing system of  claim 1 , wherein the process further comprises:
 monitoring a rotation of the first hand of the user such that rotation of the first hand of the user is mapped to rotation of at least one of the selected multiple objects; and   based on the monitoring, detecting a rotation of the first hand of the user and, in response, causing a corresponding rotation of the at least one of the selected multiple objects.   
     
     
         5 . The computing system of  claim 1 , wherein the process further comprises:
 determining which eye of the user is the user's dominant eye by one or both of:   determining which hand the user employs most often; and/or   determining how the user has positioned himself or herself in relation to past stimulus;   wherein the point at which the tip of the pyramid is placed is a point corresponding the determined dominant eye of the user.   
     
     
         6 . The computing system of  claim 1 , wherein the process further comprises:
 determining which eye of the user is the user's dominant eye by determining how the user has positioned himself or herself in relation to past stimulus;   wherein the point at which the tip of the pyramid is placed is a point corresponding the determined dominant eye of the user.   
     
     
         7 . The computing system of  claim 1 , wherein the identifying multiple objects that at least partially intersect with at least part of the pyramid comprises limiting which objects are identified to only those that are on an opposite side of the rectangle from the user. 
     
     
         8 . A computer-readable storage medium storing instructions that, when executed by a computing system, cause the computing system to perform a process comprising:
 monitoring a fist control point based on a position of a part of a first hand of the user;   monitoring a second control point based on a position of a part of a second hand of the user;   defining a rectangle with opposite corners corresponding to the fist control point and the second control point;   generating a pyramid projection, wherein a tip of the pyramid is at a point corresponding to one of the user's eyes and each of the sides of the pyramid intersect with a different one of the lines that forms the rectangle;   identifying multiple objects that at least partially intersect with at least part of the pyramid projection, either simultaneously or at different times; and   selecting the multiple objects.   
     
     
         9 . The computer-readable storage medium of  claim 8 , wherein the process is performed in response to identifying that the user performed a gesture of:
 bringing a thumb-tip on the user's first hand together with a fingertip on the user's first hand to make a first contact point;   bringing a thumb-tip on the user's second hand together with a fingertip on the user's second hand to make a second contact point;   bringing the first contact point together with the second contact point; and   pulling the first contact point away from the second contact point.   
     
     
         10 . The computer-readable storage medium of  claim 8 , wherein the process further comprises:
 in response to the selecting the multiple objects, causing a menu to be attached to the first hand of the user;   identifying an interaction, by the user's second hand, with an item on the menu; and   in response to the interaction, causing a resize or rotation of at least one of the selected multiple objects.   
     
     
         11 . The computer-readable storage medium of  claim 8 , wherein the process further comprises:
 monitoring a rotation of the first hand of the user such that rotation of the first hand of the user is mapped to rotation of at least one of the selected multiple objects; and   based on the monitoring, detecting a rotation of the first hand of the user and, in response, causing a corresponding rotation of the at least one of the selected multiple objects.   
     
     
         12 . The computer-readable storage medium of  claim 8 , wherein the process further comprises:
 determining which eye of the user is the user's dominant eye by one or both of:   determining which hand the user employs most often; and/or   determining how the user has positioned himself or herself in relation to past stimulus;   wherein the point at which the tip of the pyramid is placed is a point corresponding the determined dominant eye of the user.   
     
     
         13 . The computer-readable storage medium of  claim 8 , wherein the process further comprises:
 determining which eye of the user is the user's dominant eye by determining how the user has positioned himself or herself in relation to past stimulus;   wherein the point at which the tip of the pyramid is placed is a point corresponding the determined dominant eye of the user.   
     
     
         14 . The computer-readable storage medium of  claim 8 , wherein the identifying multiple objects that at least partially intersect with at least part of the pyramid comprises limiting which objects are identified to only those that are on an opposite side of the rectangle from the user. 
     
     
         15 . A method comprising:
 monitoring a fist control point based on a position of a part of a first hand of the user;   monitoring a second control point based on a position of a part of a second hand of the user;   defining a rectangle with opposite corners corresponding to the fist control point and the second control point;   generating a pyramid projection, wherein a tip of the pyramid is at a point corresponding to one of the user's eyes and each of the sides of the pyramid intersect with a different one of the lines that forms the rectangle;   identifying multiple objects that at least partially intersect with at least part of the pyramid projection, either simultaneously or at different times; and   selecting the multiple objects.   
     
     
         16 . The method of  claim 15 , wherein the method is performed in response to identifying that the user performed a gesture of:
 bringing a thumb-tip on the user's first hand together with a fingertip on the user's first hand to make a first contact point;   bringing a thumb-tip on the user's second hand together with a fingertip on the user's second hand to make a second contact point;   bringing the first contact point together with the second contact point; and   pulling the first contact point away from the second contact point.   
     
     
         17 . The method of  claim 15  further comprising:
 in response to the selecting the multiple objects, causing a menu to be attached to the first hand of the user; 
 identifying an interaction, by the user's second hand, with an item on the menu; and 
 in response to the interaction, causing a resize or rotation of at least one of the selected multiple objects. 
 
     
     
         18 . The method of  claim 15  further comprising:
 monitoring a rotation of the first hand of the user such that rotation of the first hand of the user is mapped to rotation of at least one of the selected multiple objects; and 
 based on the monitoring, detecting a rotation of the first hand of the user and, in response, causing a corresponding rotation of the at least one of the selected multiple objects. 
 
     
     
         19 . The method of  claim 15  further comprising:
 determining which eye of the user is the user's dominant eye by one or both of: 
 determining which hand the user employs most often; and/or 
 determining how the user has positioned himself or herself in relation to past stimulus; 
 wherein the point at which the tip of the pyramid is placed is a point corresponding the determined dominant eye of the user. 
 
     
     
         20 . The method of  claim 15 , wherein the identifying multiple objects that at least partially intersect with at least part of the pyramid comprises limiting which objects are identified to only those that are on an opposite side of the rectangle from the user.