Integrating overlaid textual digital content into displayed data via graphics processing circuitry using a frame buffer

Abstract

An apparatus, method, and computer readable medium for generating and displaying a dynamic language translation overlay that include accessing a frame buffer of the GPU, analyzing, in the frame buffer of the GPU, a frame representing a section of a stream of displayed data that is being displayed by a display device, based on the analyzed frame, identifying a reference patch that includes an instruction to identify an object comprising original text, based on the instruction included in the reference patch, recognizing the original text, generating translated text, generating an overlay comprising an augmentation layer, the augmentation layer including the translated text, and overlaying the overlay, onto the displayed data such that the translated text is viewable while the original text is obscured from view.

1 . An apparatus, comprising:
 processing circuitry configured to
 identify a reference patch in a frame buffer that includes an instruction to identify first content on an original layer of displayed data that is being displayed by the apparatus, the reference patch identifying an available area in which second content is insertable in the displayed data and a screen position within the available area at which the second content is insertable into the displayed data, 
 generate an overlay positioned on top of the original layer, the overlay comprising an augmentation layer, the augmentation layer including the second content, a position of the second content on the augmentation layer determined, at least in part, by the reference patch, and 
 overlay the overlay onto the original layer of the displayed data such that the second content is viewable while the first content is obscured from view. 
   
     
     
         2 . The apparatus of  claim 1 , wherein the overlay further comprises an overlay mask that includes non-transparent pixels, the overlay mask obscuring the first content from view and being in a different area of the augmentation layer than the second content. 
     
     
         3 . The apparatus of  claim 1 , wherein
 the overlay includes a plurality of augmentation layers, and   the processing circuitry is configured to, prior to overlaying the overlay
 determine a size of the second content, 
 determine an obscuring of elements present in the displayed data which are different from the first content, and 
 adjust the overlay, which includes the plurality of augmentation layers, such that the elements present in the displayed data which are different from the first content are not obscured. 
   
     
     
         4 . The apparatus of  claim 1 , wherein
 the first content corresponds to original text,   the second content corresponds to a translation of the original text, and   the instruction included in the reference patch includes an indication of a translation language target, which is an indication of a language in which the original text is to be translated into.   
     
     
         5 . The apparatus of  claim 1 , wherein the processing circuitry is configured to recognize the first content based on the instruction included in the reference patch. 
     
     
         6 . The apparatus of  claim 1 , wherein the processing circuitry is configured to modify the overlay to allow user interaction with an interactive element in the displayed data. 
     
     
         7 . The apparatus of  claim 1 , wherein the processing circuitry generates the overlay such that the second content included in the augmentation layer is displayed in a different color or size than the first content in the original layer. 
     
     
         8 . A method, comprising:
 identifying a reference patch in a frame buffer that includes an instruction to identify first content on an original layer of displayed data that is being displayed by a display device, the reference patch identifying an available area in which second content is insertable in the displayed data and a screen position within the available area at which the second content is insertable into the displayed data;   generating an overlay positioned on top of the original layer, the overlay comprising an augmentation layer, the augmentation layer including the second content, a position of the second content on the augmentation layer determined, at least in part, by the reference patch; and   overlaying the overlay onto the original layer of the displayed data such that the second content is viewable while the first content is obscured from view.   
     
     
         9 . The method of  claim 8 , wherein the overlay further comprises an overlay mask that includes non-transparent pixels, the overlay mask obscuring the first content from view and being in a different area of the augmentation layer than the second content. 
     
     
         10 . The method of  claim 8 ,
 wherein the overlay includes a plurality of augmentation layers, and   the method further comprising, prior to overlaying the overlay
 determining a size of the second content; 
 determining an obscuring of elements present in the displayed data which are different from the first content; and 
 adjusting the overlay, which includes the plurality of augmentation layers, such that the elements present in the displayed data which are different from the first content are not obscured. 
   
     
     
         11 . The method of  claim 8 , wherein
 the first content corresponds to original text,   the second content corresponds to a translation of the original text, and   the instruction included in the reference patch includes an indication of a translation language target, which is an indication of a language in which the original text is to be translated into.   
     
     
         12 . The method of  claim 8 , further comprising:
 recognizing the first content based on the instruction included in the reference patch.   
     
     
         13 . The method of  claim 8 , further comprising:
 modifying the overlay to allow user interaction with an interactive element in the displayed data.   
     
     
         14 . The method of  claim 8 , wherein the generating generates the overlay such that the second content included in the augmentation layer is displayed in a different color or size than the first content in the original layer. 
     
     
         15 . A non-transitory computer-readable storage medium for storing computer-readable instructions that, when executed by a computer, cause the computer to perform a method, the method comprising:
 identifying a reference patch in a frame buffer that includes an instruction to identify first content on an original layer of displayed data that is being displayed by a display device, the reference patch identifying an available area in which second content is insertable in the displayed data and a screen position within the available area at which the second content is insertable into the displayed data;   generating an overlay positioned on top of the original layer, the overlay comprising an augmentation layer, the augmentation layer including the second content, a position of the second content on the augmentation layer determined, at least in part, by the reference patch; and   overlaying the overlay onto the original layer of the displayed data such that the second content is viewable while the first content is obscured from view.   
     
     
         16 . The non-transitory computer-readable storage medium of  claim 15 , wherein the overlay further comprises an overlay mask that includes non-transparent pixels, the overlay mask obscuring the first content from view and being in a different area of the augmentation layer than the second content. 
     
     
         17 . The non-transitory computer-readable storage medium of  claim 15 ,
 wherein the overlay includes a plurality of augmentation layers, and   the method further comprising, prior to overlaying the overlay
 determining a size of the second content; 
 determining an obscuring of elements present in the displayed data which are different from the first content; and 
 adjusting the overlay, which includes the plurality of augmentation layers, such that the elements present in the displayed data which are different from the first content are not obscured. 
   
     
     
         18 . The non-transitory computer-readable storage medium of  claim 15 , wherein
 the first content corresponds to original text,   the second content corresponds to a translation of the original text, and   the instruction included in the reference patch includes an indication of a translation language target, which is an indication of a language in which the original text is to be translated into.   
     
     
         19 . The non-transitory computer-readable storage medium of  claim 15 , further comprising:
 recognizing the first content based on the instruction included in the reference patch.   
     
     
         20 . The non-transitory computer-readable storage medium of  claim 15 , further comprising:
 modifying the overlay to allow user interaction with an interactive element in the displayed data.