Using natural movements of a hand-held device to manipulate digital content

Abstract

A mobile device, such as a smart phone, is provided with a camera. Digital content displayed on display screen of the mobile device may be manipulated in response to natural movements of the mobile device by a user. Motion of the mobile device is detected relative to a nearby textured surface by analyzing images of the textured surface. The displayed digital content is manipulated in response to the detected motion of the mobile device.

What is claimed is: 
     
       1. A method comprising:
 displaying, by a display screen of a mobile device, digital content, as displayed digital content; 
 receiving, by a camera of the mobile device, an image of a textured surface; 
 determining a camera pose of the camera based on the image of the textured surface, in the form of a camera matrix that defines how a point from a virtual world coordinate frame projects onto image coordinates used for the display screen of the mobile device; and 
 manipulating the displayed digital content based on the camera pose. 
 
     
     
       2. The method of  claim 1 , wherein manipulating the displayed digital content further comprises:
 defining a virtual world coordinate system that is referenced to the image of the textured surface; 
 transforming digital content to the virtual world coordinate system; and 
 tracking the camera pose periodically as the camera is moved relative to the textured surface. 
 
     
     
       3. The method of  claim 2 , wherein manipulating the displayed digital content comprises displaying, on the display screen of the mobile device, a rendered version of the transformed digital content responsive to the camera pose, wherein the digital content displayed on the display screen is manipulated in response to motion of the mobile device. 
     
     
       4. The method of  claim 1 , wherein manipulating the displayed digital content includes zooming inward in response to the mobile device moving closer to the textured surface and zooming outward in response to mobile device moving further from the textured surface. 
     
     
       5. The method of  claim 1 , wherein manipulating the displayed digital content includes zooming inward in response to the mobile device being rotated in a first direction relative to the textured surface and zooming outward in response to mobile device being rotated in a different direction relative to the textured surface. 
     
     
       6. The method of  claim 1 , wherein manipulating the displayed digital content includes extending a window that holds the digital content, such that the window is extended in a direction corresponding to motion of the mobile device relative to the textured surface. 
     
     
       7. The method of  claim 1 , wherein manipulating the displayed digital content includes warping the perspective view of the digital content in response to rotating the mobile device relative to the textured surface. 
     
     
       8. The method of  claim 1 , wherein manipulating the displayed digital content includes selecting a window for display from a plurality of windows in response to pointing the mobile device towards a particular region of the textured surface. 
     
     
       9. The method of  claim 8 , further comprising identifying the particular region based on surface parameters of the textured surface. 
     
     
       10. The method of  claim 1 , wherein manipulating the displayed digital content includes selecting a particular object within the digital content by hovering the motion of the mobile device in alignment with the particular object. 
     
     
       11. A mobile device, comprising:
 a camera configured to receive an image of a textured surface; 
 a display screen configured to display digital content, as displayed digital content; and 
 a processor coupled to the camera and to the display screen, the processor configured to:
 determine a camera pose of the camera based on the image of the textured surface, in the form of a camera matrix that defines how a point from a virtual world coordinate frame projects onto image coordinates used for the display screen; and 
 manipulate the displayed digital content based on the camera pose. 
 
 
     
     
       12. The mobile device of  claim 11 , wherein the processor is further configured to:
 define a virtual world coordinate system that is referenced to the image of the textured surface; 
 transform digital content to the virtual world coordinate system; and 
 track the camera pose periodically as the camera is moved relative to the textured surface; and 
 wherein the display screen is further configured to display a rendered version of the transformed digital content responsive to the camera pose, such that the digital content displayed on the display screen is manipulated in response to motion of the mobile device. 
 
     
     
       13. The mobile device of  claim 11 , wherein the processor is further configured to zoom inward in response to the mobile device moving closer to the textured surface and zoom outward in response to mobile device moving further from the textured surface. 
     
     
       14. The mobile device of  claim 11 , wherein the processor is further configured to zoom inward in response to the mobile device being rotated in a first direction relative to the textured surface and zoom outward in response to mobile device being rotated in a different direction relative to the textured surface. 
     
     
       15. The mobile device of  claim 11 , wherein the processor is further configured to extend a window that holds the digital content, such that the window is extended in a direction corresponding to motion of the mobile device relative to the textured surface. 
     
     
       16. The mobile device of  claim 11 , wherein the processor is further configured to warp the perspective view of the digital content in response to rotating the mobile device relative to the textured surface. 
     
     
       17. The mobile device of  claim 11 , wherein the processor is further configured to select a window for display from a plurality of windows in response to pointing the mobile device at a particular region of the textured surface. 
     
     
       18. The mobile device of  claim 17 , wherein the processor is further configured to identify the particular region based on surface parameters of the textured surface. 
     
     
       19. A non-transitory computer readable storage medium storing a program for execution by a processor of a mobile device, the program including instructions to:
 display, by a display screen of the mobile device, digital content as displayed digital content; 
 receive, by a camera of the mobile device, an image of a textured surface; 
 determine a camera pose of the camera based on the image of the textured surface, in the form of a camera matrix that defines how a point from a virtual world coordinate frame projects onto image coordinates used for the display screen of the mobile device; and 
 manipulate the displayed digital content based on the camera pose. 
 
     
     
       20. The non-transitory computer readable storage medium of  claim 19 , wherein the instructions to manipulate the displayed digital content further comprise instructions to:
 define a virtual world coordinate system that is referenced to the image of the textured surface; 
 transform digital content to the virtual world coordinate system; and 
 track the camera pose periodically as the camera is moved relative to the textured surface; and 
 display on the display screen of the mobile device a rendered version of the transformed digital content responsive to the camera pose, such that the digital content displayed on the display screen is manipulated in response to motion of the mobile device.