Digital image transcription and manipulation

Abstract

Computer servers configured to perform digital image processing are discloses herein. In one embodiment, upon receiving a transcription command, a computer server performs text recognition based on patterns in a digital image to generate digital text data corresponding to the digital image. The computer server can then determine a content format of the digital text data based on the imported digital image and automatically apply the determined content format to the generated digital text data. The digital data can then be inserted into an electronic document.

We claim: 
     
       1. A method of automatic digital image transcription in an electronic document, the method comprising:
 importing, a digital image into a body section of an electronic document being composed in an email client at a client device; 
 receiving, at the client device, a transcription command to automatically transcribe the imported digital image in the electronic document; and 
 in response to receiving the transcription command and before the electronic document is transmitted,
 transmitting a copy of the imported digital image to a server interconnected to the client device via a computer network; 
 performing, at the server, text recognition based on patterns in the imported digital image to generate digital text data corresponding to the copy of the imported digital image; 
 determining, at the server, a content format of the digital text data based on the imported digital image by recognizing one or more keywords or punctuations individually corresponding to a text structure; 
 automatically applying, at the server, the determined content format to the generated digital text data such that the generated digital text data has the text structure corresponding to the recognized one or more keywords or punctuations, thereby typesetting the digital text data; 
 providing a copy of the typeset digital text data to the client device from the server via the computer network; and 
 inserting and surfacing, at the client device, the provided copy of the typeset digital text data into a copy of the electronic document. 
 
 
     
     
       2. The method of  claim 1 , further comprising:
 receiving a request from a user to import the digital image into the electronic document; 
 in response to receiving the request, importing the digital image into a body section of the electronic document; and 
 wherein performing text recognition includes performing text recognition based on patterns in the imported digital image in the body section of the electronic document to generate digital text data corresponding to the imported digital image. 
 
     
     
       3. The method of  claim 1  wherein performing pattern recognition includes performing text recognition based on pattern models to recognize handwritten patterns of characters in the imported digital image to generate the digital text data corresponding to the imported digital image. 
     
     
       4. The method of  claim 1  wherein:
 determining the content format includes determining one or more of a list, a table, or a paragraph according to which the digital text data is to be arranged; and 
 automatically applying the determined content format includes automatically arranging the digital text data into one or more of the list, the table, or the paragraph. 
 
     
     
       5. The method of  claim 1  wherein:
 determining the content format includes determining one or more of a list, a table, or a paragraph according to which the digital text data is to be arranged based on one or more of keywords, punctuations, or content appearance; and 
 automatically applying the determined content format includes automatically arranging the digital text data into one or more of the list, the table, or the paragraph based on the one or more of keywords, punctuations, or content appearance. 
 
     
     
       6. The method of  claim 1  wherein:
 the determined content format is a list; and 
 automatically applying the determined content format includes:
 automatically arranging the digital text data into a column; and 
 inserting one or more of bullet points or numerical headings in front of each entry in the column. 
 
 
     
     
       7. The method of  claim 1  wherein inserting the typeset digital text data includes inserting the typeset digital text data into the copy of the electronic document being composed in place of the imported digital image. 
     
     
       8. The method of  claim 1  wherein inserting the typeset digital text data includes inserting the typeset digital text data into the copy of the electronic document being composed in addition to the imported digital image. 
     
     
       9. The method of  claim 1 , further comprising:
 subsequent to inserting the typeset digital text data into the copy of the electronic document being composed,
 surfacing the inserted typeset digital text data to a user; and 
 receiving one or more edits to the surfaced typeset digital text data. 
 
 
     
     
       10. A computing device for processing electronic documents, the computing device being interconnected to a remote server via a computer network, wherein the computing device comprising:
 a processor; and 
 a memory containing instructions executable by the processor to cause the computing device to:
 receive a command to transcribe an imported digital image in an electronic document at the computing device; and 
 in response to receiving the command and before the electronic document is transmitted, transmitting, from the computing device and via the computer network, a transcription request along with a copy of the imported digital image to the remote server, wherein the remote server is configured to:
 perform text recognition based on patterns in a received copy of the imported digital image to generate digital text data corresponding to the received copy of the imported digital image; 
 recognize one or more keywords or punctuations in the generated digital text data individually corresponding to a text structure; 
 automatically format the generated digital text data such that the generated digital text data has the text structure corresponding to the recognized one or more keywords or punctuations; and 
 transmit, via the computer network, the digital text data with the text structure to the computing device; and 
 
 upon receiving the copy of the digital text data with the text structure, insert the received copy of the digital text data with the text structure into the electronic document. 
 
 
     
     
       11. The computing device of  claim 10  wherein:
 the remote server is also configured to:
 recognize a content format of the digital text data based on the digital image; and 
 automatically format the digital text data using the recognized content format to generate typeset digital text data; and 
 transmit the typeset digital text data to the computing device. 
 
 
     
     
       12. The computing device of  claim 10  wherein:
 the digital image contains data representing handwritten characters; and 
 the digital text data includes data transcribed from the handwritten characters. 
 
     
     
       13. The computing device of  claim 10  wherein:
 the digital image contains data representing handwritten characters in a list; and 
 the digital text data includes data transcribed from the handwritten characters in the list. 
 
     
     
       14. The computing device of  claim 10  wherein:
 the digital image contains data representing handwritten characters in a table; and 
 the digital text data includes data transcribed from the handwritten characters in the table. 
 
     
     
       15. The computing device of  claim 10  wherein the imported digital image is inserted into the electronic document along with the digital text data. 
     
     
       16. The computing device of  claim 10  wherein the received digital text data is inserted into the electronic document in place of the imported digital image. 
     
     
       17. A method of automatic digital image transcription in an electronic document, the method comprising:
 receiving, at a server, (i) a transcription command from a user at a client device to transcribe an imported digital image in the electronic document at the client device and (ii) a copy of the imported digital image; and 
 in response to receiving the transcription command from the client device, at the server,
 performing text recognition based on patterns in the received copy of the imported digital image to generate digital text data corresponding to the received copy of the imported digital image; 
 determining a content format of the digital text data based on the received copy of the imported digital image by recognizing one or more keywords or punctuations individually corresponding to a text structure; 
 automatically applying the determined content format to the generated digital text data such that the generated digital text data has the text structure corresponding to the recognized one or more keywords or punctuations, thereby typesetting the digital text data; 
 inserting the typeset digital text data into a server copy of the electronic document at the server; and 
 transmitting a copy of the inserted typeset digital text data in the server copy to the client device. 
 
 
     
     
       18. The method of  claim 17  wherein:
 determining the content format includes determining one or more of a list, a table, or a paragraph according to which the digital text data is to be arranged; and 
 automatically applying the determined content format includes automatically arranging the digital text data into one or more of the list, the table, or the paragraph. 
 
     
     
       19. The method of  claim 17  wherein:
 determining the content format includes determining one or more of a list, a table, or a paragraph according to which the digital text data is to be arranged based on one or more of keywords, punctuations, or content appearance; and 
 automatically applying the determined content format includes automatically arranging the digital text data into one or more of the list, the table, or the paragraph based on the one or more of keywords, punctuations, or content appearance. 
 
     
     
       20. The method of  claim 17  wherein:
 the determined content format is a list; and 
 automatically applying the determined content format includes:
 automatically arranging the digital text data into a column; and 
 inserting one or more of bullet points or numerical headings in front of each entry in the column.