Document Mode Processing For Portable Reading Machine Enabling Document Navigation

Abstract

Controlling a reading machine while reading a document to a user by receiving an image of a document, accessing a knowledge base that provides data that identifies sections in the document and processing user commands to select a section of the document. The reading machine applies text-to-speech to a text file that corresponds to the selected section of the document, to read the selected section of the document aloud to the user.

1 - 20 . (canceled) 
     
     
         21 . A method of operating a portable device, the method comprising:
 receiving by a processor in the portable device data that corresponds to an image captured by an image input device mounted on a set of eyeglasses, the image input device in communication with the portable device;   processing by the processor the image to recognize a user-initiated gesture in a scene captured in the image;   determining by the processor based on the recognized user-initiated gesture a function to perform by the portable device; and   performing the determined function by the portable device according to the recognized user-initiated gesture.   
     
     
         22 . The method of  claim 21  wherein the user selects a gesturing mode to configure the processor, the method further comprising:
 determining by the processor that a pointer for providing the gesture was not detected; and 
 generating by the processor a notification to the user that the pointer was not detected by the portable device. 
 
     
     
         23 . The method of  claim 22  wherein a directed reading mode is selected by the user. 
     
     
         24 . The method of  claim 21  wherein the camera mounted on the eyeglasses is a low resolution camera for gesture processing and the portable device includes a separate camera that captures high-resolution images for text recognition processing. 
     
     
         25 . The method of  claim 24  wherein processing comprises:
 applying pattern-recognition processing to low resolution images captured by cameras on the eyeglasses to detect one of a set of pre-defined gestures that are interpreted by the portable device, as commands to the portable device. 
 
     
     
         26 . The method of  claim 24  wherein the eyeglasses have scanning sensors that produce signals that are translated into signals that allows the user to control the direction of scanning through motion of the user's head. 
     
     
         27 . The method of  claim 21  wherein gesturing involves a gesturing item. 
     
     
         28 . The method of  claim 27  wherein the gesturing item is a portion of the user's hand or an item held by the user and the gesturing indicates the command from the user to the portable device. 
     
     
         29 . The method of  claim 21  wherein commands include: read text, describe scene, spell word, speak faster/slower, skip sentence or paragraph, zoom in or out, and read named part of template and wherein the commands are pre-associated with defined gestures. 
     
     
         30 . The method of  claim 26  wherein the command to the portable device to navigate through a hierarchical document, by processing different sections of the document for speech synthesis, is in an order determine by the command. 
     
     
         31 . The method of  claim 21  further comprising:
 receiving a series of images from the image input device; 
 processing by the processor in the portable device, the images to extract a series of low resolution images; and 
 processing by the processor in the portable device the series of low resolution images to detect the presence of the user-initiated gesture in the series of images. 
 
     
     
         32 . A portable device comprises:
 a processor;   memory in communication with the processor, the processor configured to:
 receive data that corresponds to an image captured by an image input device mounted on a set of eyeglasses, the image input device in communication with the portable device; 
 process the image to recognize a user-initiated gesture in a scene captured in the image; 
 determine based on the recognized user-initiated gesture a function to perform by the portable device; and 
 perform the determined function by the portable device according to the recognized user-initiated gesture. 
   
     
     
         33 . The portable device of  claim 32  wherein the user selects a gesturing mode to configure the processor for gesture processing, the method further comprising:
 determine that a pointer for providing the gesture was not detected; and 
 provide a notification to the user that the pointer was not detected by the portable device. 
 
     
     
         34 . The portable device of  claim 32  wherein a directed reading mode is selected by the user. 
     
     
         35 . The portable device of  claim 32  wherein the camera mounted on the eyeglasses is a low resolution camera for gesture processing and the portable device includes a separate camera that captures high-resolution images for text recognition processing. 
     
     
         36 . The portable device of  claim 35  wherein the processor is further configured to:
 apply pattern-recognition processing to low resolution images captured by low resolution camera on the eyeglasses to detect one of a set of pre-defined gestures that are interpreted by the portable device, as commands to the portable device. 
 
     
     
         37 . The portable device of  claim 32  wherein the commands include read text, describe scene, spell word, speak faster/slower, skip sentence or paragraph, zoom in or out, and read named part of template and wherein the commands are pre-associated with defined gestures. 
     
     
         38 . The portable device of  claim 32  wherein the processor is further configured to:
 receive a series of images from the image input device; 
 process the images to extract a series of low resolution images; and 
 process the series of low resolution images to detect the presence of the user-initiated gesture in the series of images. 
 
     
     
         39 . A computer program product tangibly stored on a computer readable hardware storage device for controlling a portable electronic device, the computer program product comprises instructions for causing a processor to:
 receive data that corresponds to an image captured by an image input device mounted on a set of eyeglasses, the image input device in communication with the portable device;   process the image to recognize a user-initiated gesture in a scene captured in the image;   determine based on the recognized user-initiated gesture a function to perform by the portable device; and   perform the determined function by the portable device according to the recognized user-initiated gesture.   
     
     
         40 . The computer program product of  claim 39  further comprises instructions to:
 apply pattern-recognition processing to low resolution images captured by low resolution camera on the eyeglasses to detect one of a set of pre-defined gestures that are interpreted by the portable device, as commands to the portable device.