Augmented reality hand gesture recognition systems

Abstract

A method, system, apparatus, and/or device for detecting pinch gestures in an augmented reality environment. The method, system, apparatus, and/or device may include: a wearable display, a sensor, and a processing device. The wearable display may be configured to attach to a head of a user and display an augmented reality environment to the user. The sensor may be configured to detect a position of a first digit of a hand of the user and detect a position of a second digit of the hand of the user. The processing device may be configured to: identify a first fingertip of the first digit; identify a second fingertip of the second digit; determine that the first fingertip and the second fingertip are in an open pinch position at a first point in time; and display a cursor at a midpoint between the first fingertip and the second fingertip.

1 . A device, comprising:
 a wearable display configured to:   attach to a head of a user; and   display an augmented reality environment to the user, wherein the augmented reality environment includes a virtual object;   a sensor configured to:   detect a position of a first digit of a first hand of the user; and   detect a position of a second digit of the first hand of the user;   a processing device coupled to the wearable display and the sensor, wherein the processing device is configured to:   identify a first fingertip of the first digit;   identify a second fingertip of the second digit;   determine that the first fingertip and the second fingertip are in an open pinch position at a first point in time;   determine that the first fingertip and the second fingertip are in a closed pinch position at a second point in time, wherein the second point in time is after the first point in time;   determine that the first fingertip and the second fingertip are in the closed pinch position for a first threshold period of time; and   in response to determining that the first fingertip and the second fingertip are in the closed pinch position for the first threshold period of time, execute a tap instruction associated with the closed pinch position, wherein the tap instruction corresponds with a tap instruction executed on a touch sensor of another device.   
     
     
         2 . A method, comprising:
 detecting, by a sensor, a position of a first digit of a hand of a user;   detecting, by the sensor, a position of a second digit of the hand of the user;   identifying, by a processing device, a first fingertip of the first digit;   identifying, by the processing device, a second fingertip of the second digit;   determining, by the processing device, that the first fingertip and the second fingertip are in an open pinch position at a first point in time;   determining, by the processing device, that the first fingertip and the second fingertip are in a closed pinch position at a second point in time, wherein the second point in time is after the first point in time;   determining, by the processing device, that the first fingertip and the second fingertip are in the closed pinch position for a threshold period of time; and   in response to determining that the first fingertip and the second fingertip are in the closed pinch position for the threshold period of time, executing a tap instruction associated with the closed pinch position, wherein the tap instruction corresponds with a tap instruction executed on a touch sensor of another device.   
     
     
         3 . A device, comprising:
 a wearable display configured to:   attach to a head of a user; and   display an augmented reality environment to the user, wherein the augmented reality environment includes a virtual object;   a sensor configured to:   detect a position of a first digit of a hand of the user; and   detect a position of a second digit of the hand of the user;   a processing device coupled to the wearable display and the sensor, wherein the processing device is configured to:   identify a first fingertip of the first digit;   identify a second fingertip of the second digit;   determine that the first fingertip and the second fingertip are in an open pinch position at a first point in time; and   display, by the wearable display, a cursor at a midpoint between the first fingertip and the second fingertip.