Gesture-based GUI for computing devices

Abstract

A computing device provides augmented reality images of an environment in which the computing device is used. The computing device is further configured to display a graphical user interface for interacting with the computing device. The graphical user interface may be displayed according to one or more configured graphical user interface layouts. The computing device further includes an inertial measurement unit, which provides input for interacting with one or more portions of the graphical user interface. As a user of the computing device moves the computing device, corresponding graphical changes are made to the displayed graphical user interface. In this way, by moving the computing device, the user is able to interact with, and provide input to, the computing device.

We claim: 
     
       1. A computing device having a display that displays a graphical user interface for interacting with the computing device, the computing device comprising:
 a machine-readable memory storing computer-executable instructions; and 
 at least one hardware processor in communication with the machine-readable memory that, when the computer-executable instructions are executed, configures the computing device to perform a plurality of operations, the plurality of operations comprising: 
 displaying, on a display in communication with the at least one hardware processor, a graphical user interface, the graphical user interface having at least one actuatable portion associated with an application executable by the computing device, wherein the at least one actuatable portion comprises an edge of the displayed graphical user interface; 
 acquiring, by at least one inertial measurement unit, a plurality of measurements indicating user movement of the computing device; 
 converting the plurality of measurements to obtain a plurality of vectors, wherein at least one vector of the plurality of vectors is associated with at least three axes of movement; 
 displaying a position indicator that moves in response to the plurality of vectors; 
 determining that the position indicator is at a predetermined distance from the edge of the at least one actuatable portion; 
 changing an appearance of the actuatable portion in response to the determination that the position indicator is at the predetermined distance from the edge of the at least one actuatable portion; 
 determining whether the position indicator has traversed the edge of the at least one actuatable portion; 
 determining a command to perform with the displayed graphical user interface in response to: 
 a determination that the position indicator has traversed the edge of the at least one actuatable portion; and 
 a determination that the position indicator traversed the edge of the at least one actuatable portion within a predetermined timing threshold of when the position indicator reached the predetermined distance; and 
 performing the determined command with the displayed graphical user interface. 
 
     
     
       2. The computing device of  claim 1 , wherein the plurality of operations further comprise:
 comparing one or more values of the plurality of vectors with a plurality of corresponding thresholds, wherein at least one of the plurality of thresholds is associated with the command to perform; and 
 the determining of the command to perform is further based on the comparison. 
 
     
     
       3. The computing device of  claim 1 , wherein the at least one actuatable portion is associated with a plurality of activity states, and the command to perform causes the actuatable portion to change from a first activity state to a second activity state. 
     
     
       4. The computing device of  claim 3 , wherein:
 the change from the first activity state to the second activity state causes a corresponding change in at least one actuatable portion attribute associated with the at least one actuatable portion. 
 
     
     
       5. The computing device of  claim 1 , wherein the command comprises:
 moving the position indicator to a predetermined area of the at least one actuatable portion; and 
 executing at least one application associated with the at least one actuatable portion based on the movement of the position indicator to the predetermined area. 
 
     
     
       6. The computing device of  claim 1 , wherein:
 the determined command causes a change in an appearance of the actuatable portion based on the plurality of vectors and the edge. 
 
     
     
       7. A method for displaying a graphical user interface on a display of a computing device, the method comprising:
 displaying, on a display in communication with the at least one hardware processor, a graphical user interface, the graphical user interface having at least one actuatable portion associated with an application executable by the computing device, wherein the at least one actuatable portion comprises an edge of the displayed graphical user interface; 
 acquiring, by at least one inertial measurement unit, a plurality of measurements indicating user movement of the computing device; 
 converting, by the at least one hardware processor, the plurality of measurements to obtain a plurality of vectors, wherein at least one vector of the plurality of vectors is associated with at least three axes of movement; 
 displaying a position indicator that moves in response to the plurality of vectors; 
 determining that the position indicator is at a predetermined distance from the edge of the at least one actuatable portion; 
 changing an appearance of the actuatable portion in response to the determination that the position indicator is at the predetermined distance from the edge of the at least one actuatable portion; 
 determining whether the position indicator has traversed the edge of the at least one actuatable portion; 
 determining, by the at least one hardware processor, a command to perform with the displayed graphical user interface in response to: 
 a determination that the position indicator has traversed the edge of the at least one actuatable portion; and 
 a determination that the position indicator traversed the edge of the at least one actuatable portion within a predetermined timing threshold of when the position indicator reached the predetermined distance; and 
 performing, by the at least one hardware processor, the determined command with the displayed graphical user interface. 
 
     
     
       8. The method of  claim 7 , wherein the method further comprises:
 comparing, by the at least one hardware processor, one or more values of the plurality of vectors with a plurality of corresponding thresholds, at least one of the thresholds selected from the plurality of thresholds associated with the command to perform; and 
 the determining of the command to perform is further based on the comparison. 
 
     
     
       9. The method of  claim 7 , wherein the at least one actuatable portion is associated with a plurality of activity states, and the command to perform causes the actuatable portion to change from a first activity state to a second activity state. 
     
     
       10. The method of  claim 9 , wherein:
 the change from the first activity state to the second activity state causes a corresponding change in at least one actuatable portion attribute associated with the at least one actuatable portion. 
 
     
     
       11. The method of  claim 7 , wherein the command comprises:
 moving, by the at least one hardware processor, the position indicator to a predetermined area of the at least one actuatable portion; and 
 executing, by the at least one processor, at least one application associated with the at least one actuatable portion based on the movement of the position indicator to the predetermined area. 
 
     
     
       12. The method of  claim 7 , wherein:
 the determined command causes a change in an appearance of the actuatable portion based on the plurality of vectors and the edge. 
 
     
     
       13. A non-transitory, computer-readable medium having computer-executable instructions stored thereon that, when executed by one or more hardware processors, cause a computing device to perform a plurality of operations comprising:
 displaying, on a display in communication with the at least one hardware processor, a graphical user interface, the graphical user interface having at least one actuatable portion associated with an application executable by a computing device, wherein the at least one actuatable portion comprises an edge of the displayed graphical user interface; 
 acquiring, by at least one inertial measurement unit, a plurality of measurements indicating user movement of the computing device; 
 converting the plurality of measurements to obtain a plurality of vectors, wherein at least one vector of the plurality of vectors is associated with at least three axes of movement; 
 displaying a position indicator that moves in response to the plurality of vectors; 
 determine that the position indicator is at a predetermined distance from the edge of the at least one actuatable portion; 
 changing an appearance of the actuatable portion in response to the determination that the position indicator is at the predetermined distance from the edge of the at least one actuatable portion; 
 determining whether the position indicator has traversed the edge of the at least one actuatable portion; 
 determining a command to perform with the displayed graphical user interface in response to: 
 a determination that the position indicator has traversed the edge of the at least one actuatable portion within a predetermined timing of when the position indicator reached the predetermined distance; and 
 performing the determined command with the displayed graphical user interface. 
 
     
     
       14. The non-transitory, computer-readable medium of  claim 13 , wherein the plurality of operations further comprise:
 comparing one or more values of the plurality of vectors with a plurality of corresponding thresholds, at least one of the thresholds selected from the plurality of thresholds associated with the command to perform; and 
 the determining of the command to perform is further based on the comparison. 
 
     
     
       15. The non-transitory, computer-readable medium of  claim 13 , wherein the at least one actuatable portion is associated with a plurality of activity states, and the command to perform causes the actuatable portion to change from a first activity state to a second activity state, wherein the change from the first activity state to the second activity state causes a corresponding change in at least one actuatable portion attribute associated with the at least one actuatable portion. 
     
     
       16. The non-transitory, computer-readable medium of  claim 13 , wherein the command comprises:
 moving the position indicator to a predetermined area of the at least one actuatable portion; and 
 executing at least one application associated with the at least one actuatable portion based on the movement of the position indicator to the predetermined area. 
 
     
     
       17. The non-transitory, computer-readable medium of  claim 16 , wherein:
 the at least one actuatable portion comprises a plurality of edges, at least one edge being associated with the execution of the at least one application; and 
 the predetermined area comprises the at least one edge. 
 
     
     
       18. The non-transitory, computer-readable medium of  claim 13 , wherein:
 the determined command causes a change in an appearance of the actuatable portion based on the plurality of vectors and the at least one edge.