Interacting with 2D content on 3D surfaces

Abstract

Various technologies and techniques are disclosed that enable interaction with 2D content placed on a 3D surface. The system determines where relative to a 3D surface an input device is located. If the input device is hitting a 3D surface, a hidden content in 2D is positioned so that a point representing the area hit on the 3D surface lines up with a corresponding point on the hidden content in 2D. For example, when a request is received for the input device position when an input device is detected at a location in a scene, the 3D surface is projected into two dimensions. A closest point is calculated on the projected. 3D surface to a 2D location of the input device. The closest point is provided in response to be used in positioning the hidden content with the corresponding point of the 3D surface.

1 . A computer-readable medium having computer-executable instructions for causing a computer to perform steps comprising:
 determine where relative to a 3D surface an input device is located at; and   if the input device is hitting the 3D surface, position a hidden content in 2D so that a point representing the area hit on the 3D surface lines up with a corresponding point on the hidden content in 2D.   
   
   
       2 . The computer-readable medium of  claim 1 , further having computer-executable instructions for causing a computer to perform the step comprising:
 render the hidden content inactive if the input device did not hit the 3D surface.   
   
   
       3 . The computer-readable medium of  claim 1 , further having computer-executable instructions for causing a computer to perform the step comprising:
 determine that there is a need to update the hidden content before determining where relative to the 3D surface the input device is located.   
   
   
       4 . A method for providing input device location with respect to a 3D surface comprising the steps of:
 receiving a request for an input device position when an input device is detected at a location in a scene;   projecting a 3D surface into two dimensions in the scene;   calculating a closest point on the projected 3D surface to a 2D location of the input device; and   returning the closest point in response to the request.   
   
   
       5 . The method of  claim 4 , wherein the input device is a mouse. 
   
   
       6 . The method of  claim 4 , wherein the input device is a stylus. 
   
   
       7 . The method of  claim 4 , wherein the request is received from a region. 
   
   
       8 . The method of  claim 7 , wherein the region is an arbitrary 3D geometry. 
   
   
       9 . A computer-readable medium having computer-executable instructions for causing a computer to perform the steps recited in  claim 4 . 
   
   
       10 . A method for enabling interaction with 2D content placed on a 3D surface comprising the steps of:
 determining that a hidden content in 2D that is in a 3D scene needs updated;   determining a location of the input device in the 3D scene; and   if a 3D surface in the 3D scene does not have capture, then determining if the input device hit the 3D surface in the 3D scene, and if the input device did hit the 3D surface, then using texture coordinates on a 3D triangle to determine what point of a plurality of points was hit on the hidden content in 2D, and moving the hidden content to a position such that the hidden content lines up with a corresponding point on the 3D surface.   
   
   
       11 . The method of  claim 10 , further comprising:
 if the 3D surface in the 3D scene has capture, then determining if the input device hit the 3D surface with a capture content.   
   
   
       12 . The method of  claim 11 , further comprising:
 if the 3D surface in the 3D scene has capture, and if the input device is determined to hit the 3D surface with the capture content, then using texture coordinates on the 3D triangle to determine what point of the plurality of points was hit on the hidden content in 2D, and moving the hidden content to the position such that the hidden content lines up with the corresponding point on the 3D surface.   
   
   
       13 . The method of  claim 11 , further comprising:
 if the 3D surface in the 3D scene has capture, and if the input device is determined to not hit the 3D surface with the capture content, then computing the boundary of the capture content, finding a closest point on the boundary to the location of the input device, and placing the closest point on the boundary under the location of the input device.   
   
   
       14 . The method of  claim 10 , wherein the hidden content is determined to need updated upon the occurrence of an on_move event from the input device. 
   
   
       15 . The method of  claim 10 , wherein the input device is a mouse. 
   
   
       16 . The method of  claim 10 , wherein the input device is a stylus. 
   
   
       17 . The method of  claim 10 , wherein the need to update the hidden content is determined upon receiving a request from a region for the location of the input device. 
   
   
       18 . The method of  claim 17 , wherein the region is an arbitrary 3D geometry. 
   
   
       19 . The method of  claim 10 , further comprising:
 if the 3D surface in the 3D scene does not have capture, and if the input device is determined to not hit the 3D surface in the 3D scene, then moving the hidden content away from the location of the input device.   
   
   
       20 . A computer-readable medium having computer-executable instructions for causing a computer to perform the steps recited in  claim 10 .