System to augment a visual data stream with user-specific content

Abstract

User profile information for many users may be stored in a user profile database associated with a server computer connected to a network. Based on information received from a mobile computing device, the server computer selects augment information from an augment information database coupled with the user profile database. The information received from the mobile computing device is related to frames of a video stream being captured by the mobile computing device and geographical information associated with these frames. The augment information is also selected based on the user profile associated with the mobile computing device. The augment information is to be transmitted from the server computer to the mobile computing device using wireless communication and is used by the mobile computing device to overlay the frames of the video stream being captured by the mobile computing device.

1 - 20 . (canceled) 
     
     
         21 . A method for adding relevant augment information to a video stream, the method comprising:
 accessing a first frame and a second frame of a video stream, wherein the first frame comprises objects and the second frame comprises objects;   relating patterns in the first frame and the second frame to determine points of interest contained within each of the first frame and the second frame;   receiving characteristics information of the points of interest included in the first frame and the second frame, wherein the characteristics information of the points of interest include geometric shapes of the objects contained in the first frame and geometric shapes of the objects contained in the second frame;   selecting relevant augment information from an augment information database based upon the received characteristics information; and   adding, by a processor, the selected relevant augment information to the accessed video stream.   
     
     
         22 . The method according to  claim 21 , further comprising:
 decomposing the objects contained in the first frame to form the geometric shapes of the objects contained in the first frame; and   decomposing the objects contained in the second frame to form the geometric shapes of the objects contained in the second frame.   
     
     
         23 . The method according to  claim 21 , wherein receiving characteristics information of the points of interest included in the first frame and the second frame further comprises receiving characteristics information that include other distinctive visual features of the first frame and the second frame. 
     
     
         24 . The method according to  claim 21 , wherein selecting relevant augment information further comprises matching the characteristics information with characteristics of objects that are stored in an object database to determine the points of interest captured in the video stream. 
     
     
         25 . The method according to  claim 21 , wherein the video stream comprises a video stream being captured by a mobile computing device, wherein the mobile computing device has identity information that corresponds to a user profile associated with the mobile computing device. 
     
     
         26 . The method according to  claim 25 , further comprising:
 receiving geographic information associated with the mobile computing device, wherein the geographic information pertains to a location at which the video stream is being captured;   combining the characteristics information and the geographic information; and   matching the combined characteristics information and the geographic information with characteristics of objects that are stored in an object database to identify the points of interest captured in the video stream.   
     
     
         27 . The method according to  claim 25 , wherein selecting the relevant augment information further comprises selecting the relevant augment information based on the user profile associated with the mobile computing device. 
     
     
         28 . The method according to  claim 21 , wherein adding the selected relevant augment information to the accessed video stream further comprises adding the selected relevant augment information to enable at least one of overlaying and highlighting of the points of interest in the video stream with the relevant augment information. 
     
     
         29 . An apparatus for adding relevant augment information to a video stream, the apparatus comprising:
 a memory storing machine readable instructions, that when executed, cause the apparatus to:
 access a first frame and a second frame of a video stream, wherein the first frame comprises objects and the second frame comprises objects; 
 decompose the objects contained in the first frame and the second frame into geometric shapes; 
 relate patterns in the first frame and the second frame to determine points of interest contained within each of the first frame and the second frame; 
 receive characteristics information of the determined points of interest, wherein the characteristics information of the points of interest include the geometric shapes of the objects contained in the first frame and the second frame; 
 select relevant augment information from an augment information database based upon the received characteristics information; and 
 add the selected relevant augment information to the accessed video stream; and 
   a processor to execute the machine readable instructions.   
     
     
         30 . The apparatus according to  claim 29 , wherein the machine readable instructions, that when executed, are further to cause the apparatus to:
 receive characteristics information that include other distinctive visual features of the first frame and the second frame.   
     
     
         31 . The apparatus according to  claim 29 , wherein the machine readable instructions, that when executed, are further to cause the apparatus to:
 match the characteristics information with characteristics of objects that are stored in an object database to determine the points of interest captured in the video stream.   
     
     
         32 . The apparatus according to  claim 29 , wherein the video stream comprises a video stream being captured by a mobile computing device, wherein the mobile computing device has identity information that corresponds to a user profile associated with the mobile computing device. 
     
     
         33 . The apparatus according to  claim 32 , wherein the machine readable instructions, that when executed, are further to cause the apparatus to:
 receive geographic information associated with the mobile computing device, wherein the geographic information pertains to a location at which the video stream is being captured;   combine the characteristics information and the geographic information; and   match the combined characteristics information and the geographic information with characteristics of objects that are stored in an object database to determine the points of interest captured in the video stream.   
     
     
         34 . The apparatus according to  claim 32 , wherein, to select the relevant augment information, the machine readable instructions are further to select the relevant augment information based on the user profile associated with the mobile computing device. 
     
     
         35 . The apparatus according to  claim 29 , wherein, to add the selected relevant augment information to the accessed video stream, the machine readable instructions are further to add the selected relevant augment information to enable at least one of overlaying and highlighting of the points of interest in the video stream with the relevant augment information. 
     
     
         36 . A non-transitory machine readable storage medium on which is stored machine readable instructions that when executed by a processor are to cause the processor to:
 access a first frame and a second frame of a video stream, wherein the first frame comprises objects and the second frame comprises objects;   decompose the objects contained in the first frame and the second frame into geometric shapes;   relate patterns in the first frame and the second frame to determine points of interest contained within each of the first frame and the second frame;   receive characteristics information of the determined points of interest, wherein the characteristics information of the points of interest include the geometric shapes of the objects contained in the first frame and the second frame;   select relevant augment information from an augment information database based upon the received characteristics information; and   add the selected relevant augment information to the accessed video stream.   
     
     
         37 . The non-transitory computer readable storage medium according to  claim 36 , wherein the video stream comprises a video stream being captured by a mobile computing device, wherein the mobile computing device has identity information that corresponds to a user profile associated with the mobile computing device. 
     
     
         38 . The non-transitory computer readable storage medium according to  claim 37 , wherein the machine readable instructions are further to cause the processor to:
 receive geographic information associated with the mobile computing device, wherein the geographic information pertains to a location at which the video stream is being captured;   combine the characteristics information and the geographic information; and   match the combined characteristics information and the geographic information with characteristics of objects that are stored in an object database to determine the points of interest captured in the video stream.   
     
     
         39 . The non-transitory computer readable storage medium according to  claim 37 , wherein, to select the relevant augment information, the machine readable instructions are to cause the processor to:
 select the relevant augment information based on the user profile associated with the mobile computing device.