Techniques for machine language translation of text from an image based on non-textual context information from the image

Abstract

A computer-implemented technique can include receiving, at a server from a mobile computing device, the server having one or more processors, an image including a text. The technique can include obtaining, at the server, optical character recognition (OCR) text corresponding to the text, the OCR text having been obtained by performing OCR on the image. The technique can include identifying, at the server, non-textual context information from the image, the non-textual context information (i) representing context information other than the text itself and (ii) being indicative of a context of the image. The technique can include based on the non-textual context information, obtaining, at the server, a translation of the OCR text to a target language to obtain a translated OCR text. The technique can include outputting, from the server to the mobile computing device, the translated OCR text.

What is claimed is: 
     
         1 . A computer-implemented method, comprising:
 receiving, by a server computing system and from a client computing device, an image including a text;   obtaining, by the server computing system, an optical character recognition (OCR) text corresponding to the text, the OCR text having been obtained by performing OCR on the image;   identifying, by the server computing system, non-textual context information from the image, the non-textual context information (i) representing context information other than the text itself, (ii) being indicative of a context of the image, and (iii) including at least a font of the text;   obtaining, by the server computing system, a translated OCR text based on the non-textual context information, the translated OCR text representing a translation of the OCR text to a different target language; and   outputting, from the server computing system and to the client computing device, the translated OCR text.   
     
     
         2 . The computer-implemented method of  claim 1 , further comprising determining, by the server computing system, a type of an object that is associated with the text based on the font of the text. 
     
     
         3 . The computer-implemented method of  claim 2 , wherein obtaining the translated OCR text is based on the type of the object. 
     
     
         4 . The computer-implemented method of  claim 2 , wherein determining the type of the object is further based on a shape of the object. 
     
     
         5 . The computer-implemented method of  claim 1 , wherein obtaining the translated OCR text includes:
 obtaining, by the server computing system, a baseline translated OCR text representing a machine translation of the OCR text to the target language; and   adjusting, by the server computing system, the baseline translated OCR text based on the non-textual context information to obtain the translated OCR text.   
     
     
         6 . The computer-implemented method of  claim 1 , further comprising determining, by the server computing system, a source language of the text based on the non-textual context information, wherein obtaining the translated OCR text is further based on the source language. 
     
     
         7 . The computer-implemented method of  claim 1 , further comprising determining, by the server computing system, a type of location at which the image was captured based on the non-textual context information, wherein obtaining the translated OCR text is further based on the type of location. 
     
     
         8 . A server computing system, comprising:
 a non-transitory computer-readable medium having a set of instructions stored thereon; and   one or more processors configured to execute the set of instructions, which causes the server computing system to perform operations comprising:
 receiving, from a client computing device, an image including a text; 
 obtaining optical character recognition (OCR) text corresponding to the text, the OCR text having been obtained by performing OCR on the image; 
 identifying non-textual context information from the image, the non-textual context information (i) representing context information other than the text itself, (ii) being indicative of a context of the image, and (iii) including at least a font of the text; 
 obtaining a translated OCR text based on the non-textual context information, the translated OCR text representing a translation of the OCR text to a target language; and 
 outputting, to the client computing device, the translated OCR text. 
   
     
     
         9 . The server computing system of  claim 8 , wherein the operations further comprise determining a type of an object that is associated with the text based on the font of the text. 
     
     
         10 . The server computing system of  claim 9 , wherein obtaining the translated OCR text is based on the type of the object. 
     
     
         11 . The server computing system of  claim 9 , wherein determining the type of the object is further based on a shape of the object. 
     
     
         12 . The server computing system of  claim 8 , wherein obtaining the translated OCR text includes:
 obtaining a baseline translated OCR text representing a machine translation of the OCR text to the target language; and   adjusting the baseline translated OCR text based on the non-textual context information to obtain the translated OCR text.   
     
     
         13 . The server computing system of  claim 8 , wherein the operations further comprise determining a source language of the text based on the non-textual context information, and wherein obtaining the translated OCR text is further based on the source language. 
     
     
         14 . The server computing system of  claim 8 , wherein the operations further comprise determining a type of location at which the image was captured based on the non-textual context information, and wherein obtaining the translated OCR text is further based on the type of location. 
     
     
         15 . A non-transitory computer-readable medium having a set of instructions stored thereon that, when executed by one or more processors of a server computing system, causes the server computing system to perform operations comprising:
 receiving, from a client computing device, an image including a text;   obtaining optical character recognition (OCR) text corresponding to the text, the OCR text having been obtained by performing OCR on the image;   identifying non-textual context information from the image, the non-textual context information (i) representing context information other than the text itself, (ii) being indicative of a context of the image, and (iii) including at least a font of the text;   obtaining a translated OCR text based on the non-textual context information, the translated OCR text representing a translation of the OCR text to a target; and   outputting, to the client computing device, the translated OCR text.   
     
     
         16 . The computer-readable medium of  claim 15 , wherein the operations further comprise determining a type of an object that is associated with the text based on the font of the text. 
     
     
         17 . The computer-readable medium of  claim 16 , wherein obtaining the translated OCR text is based on the type of the object. 
     
     
         18 . The computer-readable medium of  claim 16 , wherein determining the type of the object is further based on a shape of the object. 
     
     
         19 . The computer-readable medium of  claim 15 , wherein obtaining the translated OCR text includes:
 obtaining a baseline translated OCR text representing a machine translation of the OCR text to the target language; and   adjusting the baseline translated OCR text based on the non-textual context information to obtain the translated OCR text.   
     
     
         20 . The computer-readable medium of  claim 15 , wherein the operations further comprise determining a type of location at which the image was captured based on the non-textual context information, and wherein obtaining the translated OCR text is further based on the type of location.