Text overlay techniques in realtime translation

Abstract

The technology described herein employs techniques for overlaying (superimposing) translated text on top of (over) scanned text in realtime. The technology recognizes text in an image and identifies a first language in which the text is written. The technology translates the text into a second language and generates an overlay in the second language. The overlay is then inserted into the display over the text in the image.

What is claimed is: 
     
         1 . A portable computing system, comprising:
 a camera;   a display;   a recognition system configured to recognize text in an image captured by the camera in video mode, the text of a first language;   a translator configured to translate the text of the first language into text of a second language;   an overlay algorithm configured to determine orientation and alignment between the text of the first language and the text of the second language, and to overlay the text of the second language on the text of the first language, as viewed in the display; and   a microprocessor configured to execute computer-executable instructions in a memory, the instructions executed to enable the recognition system the translator, and the overlay algorithm.   
     
     
         2 . The system of  claim 1 , wherein the recognition system, the translator, and the overlay algorithm operate to overlay the text of the second language on the text of the first language in the display, in realtime. 
     
     
         3 . The system of  claim 1 , wherein the recognition system, the translator, and the overlay algorithm enable application of graphical emphasis to visually differentiate the text of the second language from the text of the first language. 
     
     
         4 . The system of  claim 1 , wherein the recognition system is configured to match a color and font of the text of the first language to the color and font of the text of the second language presented in the overlay. 
     
     
         5 . The system of  claim 1 , wherein the second language is user selectable and direction of the text of the second language is user configurable. 
     
     
         6 . The system of  claim 1 , wherein the overlay algorithm is configured to compute a relationship of textual portions of the second language to related textual portions of the first language, and to overlay the text of the second language on the text of the first language based on the relationship. 
     
     
         7 . The system of  claim 1 , wherein the overlay algorithm is configured to obtain orientation information associated with the text of the first language in the image, the orientation information relates to overlaying the text of the second language vertically or horizontally. 
     
     
         8 . The system of  claim 1 , wherein the overlay algorithm overlays the text of the second language on the text of the first language in a direction at least one of opposite from which the first language is written and orthogonal to the direction in which the first language is normally written. 
     
     
         9 . A method, comprising acts of:
 recognizing text of a first language in an image captured by a camera in a portable computing device;   translating the text of the first language into text of a second language; and   visually overlaying the text of the second language on the text of the first language, as viewed on the image in the display.   
     
     
         10 . The method of  claim 9 , further comprising:
 creating a first bounding box that circumscribes the text of the first language and a second bounding box that circumscribes the text of the second language; and   aligning the second bounding box to the first bounding box as part of overlaying the text of the second language on the text of the first language.   
     
     
         11 . The method of  claim 9 , further comprising matching a color and font of the text of the first language to the color and font of the text of the second language presented in the overlay. 
     
     
         12 . The method of  claim 9 , further comprising overlaying the text of the second language on the text of the first language in a line-by-line or word-to-word relationship. 
     
     
         13 . The method of  claim 9 , further comprising overlaying the text of the second language on the text of the first language in a direction opposite from which the first language is written or in a direction orthogonal to the direction in which the first language is normally written. 
     
     
         14 . The method of  claim 9 , further comprising enabling toggling between the text of the second language and the text of the first language. 
     
     
         15 . The method of  claim 9 , further comprising enabling tagging of objects of the first language in the image and, translation and overlaying of text of the second language on the tagged objects. 
     
     
         16 . The method of  claim 9 , further comprising performing the acts of recognizing, translating, and visually overlaying, in realtime using a video mode of the camera. 
     
     
         17 . A method, comprising acts of:
 recognizing text in an image in realtime, using a smart phone camera operating in video mode and optical character recognition, the text of a first language;   translating the text of the first language into the text of the second language; and   outputting for display through the smartphone the image with an overlay of the text of the second language on the text of the first language.   
     
     
         18 . The method of  claim 17 , further comprising enabling selection of the first language and the second language, and configuration of a direction of the second language. 
     
     
         19 . The method of  claim 17 , further comprising further comprising matching a color and font of the text of the first language to the color and font of the text of the second language presented in the overlay. 
     
     
         20 . The method of  claim 17 , further comprising recognizing and translating a hyperlink of the first language, and overlaying a hyperlink of the second language on the hyperlink of the first language.