Enhancing tangible content on physical activity surface

Abstract

A method for enhancing tangible content on a physical activity surface is described. In an example implementation, the method includes capturing, using a video capture device of a computing device, a video stream that includes an activity scene of a physical activity surface; detecting in the video stream, using a detector executable on the computing device, a tangible content item on the physical activity surface; recognizing, from the video stream, one or more visually instructive elements in the tangible content item; determining a tangible identifier based on the one or more visually instructive elements in the tangible content item; retrieving a digital content item using the tangible identifier; and providing the digital content item on the computing device.

1 . A method comprising:
 capturing, using a video capture device of a computing device, a video stream that includes an activity scene of a physical activity surface located in front of a display of the computing device;   detecting in the video stream, using a detector executable on the computing device, a first tangible interface object on the physical activity surface and a second tangible interface object on the physical activity surface;   determining a first subject matter of the first tangible interface object;   determining that a user interacted with the second tangible interface object;   determining a second subject matter of the second tangible interface object responsive to the user interacting with the second tangible interface object;   recognizing, from the video stream, one or more visually instructive elements of the first tangible interface object;   determining a concept of the one or more visually instructive elements of the first tangible interface object based on the recognized one or more visually instructive elements of the first tangible interface object, the subject matter of the first tangible interface object, and the subject matter of the second tangible interface object;   automatically retrieving a digital content item using the determined concept of one or more visually instructive elements of the first tangible interface object; and   providing the digital content item for display in a user interface of the computing device to supplement the one or more visually instructive elements of the first tangible interface object on the physical activity surface.   
     
     
         2 . The method of  claim 1 , further comprising:
 detecting, in the video stream, a distinct visual element on the first tangible interface object; and wherein   determining the concept of the one or more visually instructive elements of the first tangible interface object includes determining the concept of the one or more visually instructive elements of the first tangible interface object based on the distinct visual element on the first tangible interface object.   
     
     
         3 . The method of  claim 2 , wherein:
 the distinct visual element includes one or more of an image, a drawing, a diagram, and a vision marker visible on the first tangible interface object.   
     
     
         4 . The method of  claim 1 , further comprising:
 recognizing, from the video stream, one or more of a document title, a page number, a section title, and a section number associated with the first tangible interface object; and wherein   determining the concept of the one or more visually instructive elements of the first tangible interface object includes determining the concept of the one or more visually instructive elements of the first tangible interface object based on one or more of the document title, the page number, the section title, and the section number associated with the first tangible interface object.   
     
     
         5 . The method of  claim 1 , further comprising:
 detecting, in the video stream, a user marking visible on the first tangible interface object;   recognizing, from the video stream, one or more characters associated with the user marking; and wherein   determining the concept of the one or more visually instructive elements of the first tangible interface object includes determining the concept of the one or more visually instructive elements of the first tangible interface object based on the one or more characters associated with the user marking.   
     
     
         6 . The method of  claim 5 , wherein:
 the user marking includes one or more of a highlighting effect and a user note created by a user on the first tangible interface object.   
     
     
         7 . The method of  claim 1 , wherein determining the second subject matter of the second tangible interface object further comprises:
 recognizing one or more visually instructive elements in the second tangible interface object and   determining the second subject matter of the second tangible interface object based on the one or more visually instructive in the second tangible interface object.   
     
     
         8 . The method of  claim 1 , wherein:
 determining the concept of the one or more visually instructive elements of the first tangible interface object includes determining a first concept and a second concept based on the one or more visually instructive elements in the first tangible interface object; and   retrieving the digital content item using the determined concept includes:
 determining a contextual relationship between the first concept and the second concept; and 
 retrieving the digital content item based on the contextual relationship between the first concept and the second concept. 
   
     
     
         9 . The method of  claim 1 , wherein:
 the computing device is placed on a stand situated on the physical activity surface; and   a field of view of the video capture device of the computing device is redirected by a camera adapter situated over the video capture device of the computing device.   
     
     
         10 . The method of  claim 1 , wherein the first tangible interface object on the physical activity surface includes an area with a first tangible content section and a second tangible content section, the method further comprising:
 detecting, using a detector executable on the computing device, an arrangement of the first tangible content section relative to the second tangible content section within the area on the physical activity surface;   determining a supplemental digital content item based on the arrangement of the first tangible content section relative to the second tangible content section; and   displaying in the user interface on the computing device, the supplemental digital content along with the digital content item.   
     
     
         11 . The method of  claim 10 , wherein the first tangible content section is a first programming tile and the second tangible content section is a second programming tile, the first programming tile being coupled with the second programming tile. 
     
     
         12 . The method of  claim 11 , wherein the supplemental digital content is an instruction set for how to use the first programming tile and the second programming tile. 
     
     
         13 . The method of  claim 12 , wherein displaying the supplemental digital content further comprises:
 automatically displaying a first digital representation of the first programming tile in the user interface;   automatically displaying a second digital representation of the second programming tile in the user interface; and   automatically displaying an animation of the first digital representation interacting with the second digital representation to provide instructions on how to arrange the first digital programming tile and the second digital programming tile on the activity surface.   
     
     
         14 . The method of  claim 1 , wherein providing the digital content item for display in the user interface on the computing device further comprises:
 generating a visualization of the digital content item for display in the user interface; and   
       displaying an animation of the digital content item appearing in the user interface. 
     
     
         15 . The method of  claim 14 , wherein the animation of the digital content item depicts a first portion of the visualization of the digital content item extending out from a bottom side of the display screen and a second portion of the visualization of the digital content item appearing as the animation causes the visualization of the digital content item to move into the user interface. 
     
     
         16 . The method of  claim 15 , wherein the animation of the digital content item corresponds to an appearance of the first tangible interface object being detected on the physical activity surface. 
     
     
         17 . The method of  claim 14 , wherein the visualization of the digital content item is selectable by a user and responsive to the digital content item being selected, the digital content item may be displayed. 
     
     
         18 . A method comprising:
 capturing, using a video capture device of a computing device, a video stream that includes an activity scene of a physical activity surface;   detecting in the video stream, using a detector executable on the computing device, a book on the physical activity surface and a tangible interface object on the physical activity scene, the book including a page visible within a field of view of the video capture device;   determining, using a processor of the computing device, an identity of the page of the book from the video stream;   determining a first subject matter of the book using the identity of the page of the book;   determining that a user interacted with the tangible interface object;   determining a second subject matter of the tangible interface object responsive to determining that the user interacted with the tangible interface object;   determining, using the processor of the computing device, a concept of the page of the book using the identity of the page;   determining, using the processor of the computing device, a digital content item based on the concept of the page of the book, the first subject matter of the book, and the second subject matter of the tangible interface object; and   presenting, in a display of the computing device, the digital content item in a graphical user interface.   
     
     
         19 . The method of  claim 18 , wherein determining the identity of the page of the book further comprises:
 detecting a page identity anchor on the page; and   matching the page identity anchor to a database of page identities.   
     
     
         20 . The method of  claim 19 , wherein the book includes a plurality of pages and each of the plurality of pages has a unique page identity anchor. 
     
     
         21 . (canceled) 
     
     
         22 . (canceled) 
     
     
         23 . (canceled) 
     
     
         24 . (canceled) 
     
     
         25 . (canceled) 
     
     
         26 . (canceled) 
     
     
         27 . (canceled) 
     
     
         28 . (canceled)