Ergonomic man-machine interface incorporating adaptive pattern recognition based control system

Abstract

An adaptive interface for a programmable system, for predicting a desired user function, based on user history, as well as machine internal status and context. The apparatus receives an input from the user and other data. A predicted input is presented for confirmation by the user, and the predictive mechanism is updated based on this feedback. Also provided is a pattern recognition system for a multimedia device, wherein a user input is matched to a video stream on a conceptual basis, allowing inexact programming of a multimedia device. The system analyzes a data stream for correspondence with a data pattern for processing and storage. The data stream is subjected to adaptive pattern recognition to extract features of interest to provide a highly compressed representation which may be efficiently processed to determine correspondence. Applications of the interface and system include a VCR, medical device, vehicle control system, audio device, environmental control system, securities trading terminal, and smart house. The system optionally includes an actuator for effecting the environment of operation, allowing closed-loop feedback operation and automated learning.

What is claimed is: 
     
       1. A programmable control responsive to an user input and a signal received from a signal source, comprising:
 a controller, operating according to a predetermined program, for receiving the user input and the signal and producing a control output, said controller producing a multivalued characterization of the signal with respect to at least one parameter;   a memory for storing data relating to an activity of the user;   a data processing system for predicting a most probable action of a user based said stored data relating to said activity of the user and said characterized signal; and   a user feedback data presenting system comprising a display device for presentation of a sequence of programming options to the user, including said most probable action of the user, in a plurality of display images, each display image differing in available programming options.   
     
     
       2. The programmable control according to  claim 1  being for performing an action based on user input and an information content of a signal received from a signal source, further comprising:
 a user controlled direct manipulation-type input device, associated with said display device, having a device output, said device output being the user input; 
 a plant capable of performing a physical action, being responsive to an actuator signal; and 
 said controller, being for receiving data from said device output of said input device and a signal received from a signal source, and displaying user feedback data on said display device, 
 said user feedback data comprising a presentation of a sequence of programming options to the user on said display device, including said most probable action of the user, in a plurality of display images, each display image differing in available programming options, said sequence of programming options including at least one sequence of options sufficient to define an operable control program, and a presentation of additional programming options if said control program is not operable. 
 
     
     
       3. The system according to  claim 1 , being for processing a program comprising program material, in response to a viewer input, comprising:
 a user input processing system for determining a viewer preference;   said controller comprises a program material processing system for characterizing the program material, as said characterized signal, based on its content;   a correlator for correlating said characterized content of the program material with said determined viewer preference to produce a correlation index; and   a processor, selectively processing the program material based on said correlation index,   wherein said viewer preference is an input to said data processing system, and   said data processing system comprises said correlator.   
     
     
       4. The system according to  claim 3 , wherein said program material is encrypted, further comprising:
 a decryption system for decrypting the program material if it is selected to produce unencrypted program material and optionally an associated decryption event;   a memory for storing data relating to the occurrence of said decryption event; and   a central database for storing data relating to the occurrence of said decryption event in association with data relating to the viewer.   
     
     
       5. The system according to  claim 3 , wherein:
 said user input processing system monitors a pattern of user activity and predicts a viewer preference;   said program material processing system comprises:   a processor for preprocessing the program material to produce a reduced data flow information signal substantially retaining information relating to an abstract information content of the program material and selectively eliminating data not relating to said abstract information content of the program material and for characterizing said information signal based on said abstract information content; and   a comparing system for determining if said correlation index is indicative of a probable high correlation between said characterization of said information signal and said viewer preference and causing said stored program material to be processed by said processor based on said determination.   
     
     
       6. The system according to  claim 3 , wherein said processor comprises an image program material storage and retrieval system. 
     
     
       7. The system according to  claim 3 , further comprising a memory for storing a characterization of the program material; an input for receiving a feedback signal from the viewer indicating a degree of agreement with said correlation index determination, wherein said feedback signal and said stored characterization are used by said user input processing system to predict a new viewer preference. 
     
     
       8. The system according to  claim 3 , wherein said a program material processing system correlates and characterizes image information, further comprising:
 means for storing template data;   means for storing the image data;   means for generating a plurality of addressable domains from the stored image data, each of the domains representing a different portion of the image information;   means for creating, from the stored image data, a plurality of addressable mapped ranges corresponding to different subsets of the stored image data, the creating means including means for executing, for each of the mapped ranges, a procedure upon the one of the subsets of the stored image data which corresponds to the mapped range;   means for assigning identifiers to corresponding ones of the mapped ranges, each of the identifiers specifying for the corresponding mapped range an address of the corresponding subset of stored image data;   means for selecting, for each of the domains, the one of the mapped ranges which most closely corresponds according to predetermined criteria;   means for representing at least a portion of the image information as a set of the identifiers of the selected mapped ranges; and   means for selecting, from the stored templates, a template which most closely corresponds to the set of identifiers representing the image information.   
     
     
       9. The system according to  claim 8  wherein said correspondence of a template and the set of identifiers is determined by a processor executing a predetermined program for executing an algorithm selected from at least one of the group consisting of algorithms for selecting a minimum Hausdorff distance between the mapped range and the domain, for selecting the highest cross-correlation of the mapped range with the domain, and for selecting the lowest mean square error of the difference between the mapped range and the domain. 
     
     
       10. The system according to  claim 9  wherein said correspondence of a template and the set of identifiers is determined by selecting, for each domain, the mapped range with the minimum modified Hausdorff distance calculated as D[db,mrb]+D[1−db,I−mrb], where D is a distance calculated between a pair of sets of data each representative of an image, db is a domain, mrb is a mapped range, I−db is the inverse of a domain, and I-mrb is an inverse of a mapped range. 
     
     
       11. The system according to  claim 8 , wherein said means for representing further comprises means for:
 (a) determining a feature of interest of the image data,   (b) selecting a mapped range corresponding to the feature of interest,   (c) storing the identifiers of the selected mapped range,   (d) selecting a further mapped range corresponding to a portion of image data having a predetermined relationship to the feature of interest, and   (e) storing the identifiers of the further mapped range.   
     
     
       12. The system according to  claim 8 , wherein said image data comprises data having three associated dimensions obtained by an image processor executing an algorithm selected from the group consisting of an algorithm for synthesizing a three dimensional representation based on a machine based prediction derived from two dimensional image data, an algorithm for synthesizing a three dimensional representation derived from a time series of pixel images, and an algorithm for synthesizing a three dimensional representation based on a image data representing a plurality of parallax views having at least two dimensions, said set of identifiers representing data in each of the three associated dimensions. 
     
     
       13. A method of programming a device, comprising the steps of:
 providing an input for user commands and feedback;   storing information relating to the user commands and feedback in a memory;   providing a data input for external information;   predicting a subsequent user command based on the stored information relating to user commands and feedback, a status of the device, and external information from the data input;   presenting the predicted user command to the user;   accepting feedback from the user relating to the predicted subsequent user command;   modifying the predicted subsequent user command based on at least the feedback; and   executing the modified predicted user command.   
     
     
       14. The method according to  claim 13 , further comprising the step of identifying a user and storing an user identifier with said stored information relating to the user commands;
 said predicting step predicting a subsequent user command based on the stored information relating to user identifier, commands and feedback, a status of the device, and external information from the data input.   
     
     
       15. A programmable device comprising:
 a user input for receiving an input variable having a path between a first input state and a second input state, said user input comprising:
 path dependent user data having transitional path information between said first input state and said second input state; and 
 path independent user data comprising information about at least one of said first state and said second state; 
 a filter, separating sad path dependent user data as user characterization data and said path independent user data as instructions; 
 a memory for storing said user characterization data; 
 a processor for executing said instructions; and 
 a feedback device, presenting information relating to said instructions and said stored user characterization data. 
   
     
     
       16. The device according to  claim 15 , further comprising:
 a hierarchical command structure of said processor, said command structure having commands of different function; and   means for predicting a probability of execution of a plurality of commands based on said input,   said feedback device presenting commands based on at least said predicted probabilities.   
     
     
       17. The programmable control according to  claim 1 , wherein the signal comprises image information. 
     
     
       18. The method according to  claim 13 , wherein said stored information comprises a set of weights of a predictive algorithm. 
     
     
       19. The method according to  claim 13 , herein said external information comprises a perceptual data stream. 
     
     
       20. The programmable device according to  claim 15 , wherein a format of information presented from said feedback device is based on said path dependent user data. 
     
     
       21. A method for receiving targeted content at a mobile device, the method comprising:
 sending a request to receive customized information at the mobile device;   in response to the request, receiving the customized information at the mobile device, wherein the customized information is dependent on position information of the mobile device, as determined by a positioning system during a time period associated with sending the request, and dependent on at least one of user profile information or preferences information stored, at least in part, in a memory device of the mobile device; and   causing a presentation of the customized information for one or more users associated with the mobile device.   
     
     
       22. The method of claim 21, wherein the customized information is an advertisement. 
     
     
       23. The method of claim 22, wherein the advertisement concerns food. 
     
     
       24. The method of claim 22, wherein the advertisement concerns fuel. 
     
     
       25. The method of claim 22, wherein the advertisement concerns lodging. 
     
     
       26. The method of claim 21, further comprising storing, in the memory device, the at least one of the user profile information or the preferences information, wherein:
 the positioning system is a geographic positioning system (GPS); and   determining further comprises determining, by the GPS, the position information of the mobile device.   
     
     
       27. The method of claim 21, wherein the positioning system is a cellular base station, and wherein determining further comprises:
 determining, by the cellular base station, the position information of the mobile device.   
     
     
       28. The method of claim 21, wherein the preferences information includes user preferences information, and wherein receiving further comprises:
 receiving one or more customized billboards at the mobile device dependent on demographics of the one or more users associated with the mobile device, dependent on the position information, and dependent on the user preferences information.   
     
     
       29. The method of claim 21, further comprising:
 receiving the customized information at the mobile device dependent on an itinerary of the one or more users; and   causing a presentation of the customized information for the one or more users associated with the mobile device based on the itinerary.   
     
     
       30. A system for receiving targeted content at a mobile device, the system comprising:
 a memory device of the mobile device configured to store at least one of user profile information or preferences information;   a positioning system configured to determine position information of the mobile device;   a transmitter configured to send a request to receive customized information at the mobile device;   a receiver configured to receive the customized information from a provider at the mobile device, wherein the customized information is dependent on the position information of the mobile device, as determined by the positioning system during a time period associated with sending the request, and dependent on at least one of the user profile information or the preferences information; and   a display device configured to display the customized information for one or more users associated with the mobile device.   
     
     
       31. The system of claim 30, wherein the customized information is an advertisement. 
     
     
       32. The system of claim 31, wherein the advertisement concerns food. 
     
     
       33. The system of claim 31, wherein the advertisement concerns fuel. 
     
     
       34. The system of claim 31, wherein the advertisement concerns lodging. 
     
     
       35. The system of claim 30, wherein:
 the positioning system is a geographic positioning system (GPS); and   the GPS is configured to determine the position information of the mobile device.   
     
     
       36. The system of claim 30, wherein:
 the positioning system is a cellular base station; and   the cellular base station is configured to determine the position information of the mobile device.   
     
     
       37. The system of claim 30, wherein:
 the preferences information includes user preferences information; and   the receiver is configured to receive, from the provider, one or more customized billboards at the mobile device dependent on demographics of the one or more users associated with the mobile device, dependent on the position information, and dependent on the user preferences information.   
     
     
       38. The system of claim 30, wherein:
 the mobile device is a cellular phone;   the cellular phone includes the display device; and   the cellular phone includes the receiver.   
     
     
       39. The system of claim 30, wherein the display device includes a touch screen. 
     
     
       40. The system of claim 30, wherein the display device includes an interactive interface configured to receive instructions from the one or more users and to display user feedback. 
     
     
       41. The system of claim 30, wherein the mobile device includes a gyroscope. 
     
     
       42. The system of claim 30, wherein the user profile information includes a demographic profile. 
     
     
       43. The system of claim 30, wherein the display device is a video display. 
     
     
       44. The system of claim 30, wherein the mobile device is configured to identify the one or more users based on a biometric. 
     
     
       45. The system of claim 44, wherein the biometric is at least one of a fingerprint, a voice sample, or a video pattern. 
     
     
       46. The system of claim 30, wherein the customized information includes an advertisement. 
     
     
       47. The system of claim 46, wherein the display device is configured to facilitate interaction between the one or more users and a service providing the advertisement. 
     
     
       48. The system of claim 30, wherein the customized information includes a plurality of advertisements. 
     
     
       49. The system of claim 30, wherein the customized information is configured to be sent via a cellular network. 
     
     
       50. The system of claim 30, wherein the display device includes a touch screen display. 
     
     
       51. The system of claim 50, wherein the touch screen display is configured to display a graphical user interface. 
     
     
       52. The system of claim 30, wherein the user profile information is based at least in part on prior actions of the one or more users.