Systems and methods to measure and enhance human engagement and cognition

Abstract

Systems and methods are described to enact machine-based, simultaneous classification of emotional and cognitive states of an individual in substantially real time. A bootstrap approach is used to identify indicators of emotional state, including boredom, engagement and feeling overwhelmed, by finding additional interactions temporally adjacent to known indicators. A similar bootstrap approach is used to identify new indicators of knowledge acquisition (or confusion), by locating frequent temporally adjacent interactions. Systems and methods may enhance teaching by identifying knowledge presentations that result in successful knowledge acquisition, and learning by allowing instructors associated with an individual to track materials that have been understood (or misunderstood). Machine-based tracking of learning may circumvent boredom by avoiding needless repetition, and facilitate teaching of synergistic topics related to materials recently understood. Systems and methods may provide for an enhanced, individualized environment for both emotional and cognitive development.

We claim: 
     
         1 . A method for a human observer of an individual to effect a target degree of emotional engagement by the individual, comprising:
 providing, in proximity to the individual, an individual electronic device that includes an individual processor, one or more individual output devices operatively coupled to the individual processor, and one or more individual sensors operatively coupled to the individual processor;   providing, in proximity to the human observer, an observer electronic device that includes an observer processor, one or more observer output devices operatively coupled to the observer processor, and one or more observer sensors operatively coupled to the observer processor;   presenting knowledge to the individual that may be unknown to the individual on the one or more individual output devices;   acquiring, from the one or more individual sensors, interaction data from an interaction between the individual and the individual electronic device related to the knowledge that was presented;   presenting the interaction data on the one or more observer output devices;   indicating, by the human observer using the one or more observer sensors, an observed degree of emotional engagement by the individual;   selecting, by one or more of the individual processor, the observer processor and one or more remote processors, an activity intended to direct the observed degree of emotional engagement toward a target degree of emotional engagement; and   presenting a human interaction entity on the one or more individual output devices to initiate the activity with the individual.   
     
     
         2 . The method of  claim 1 , wherein the individual degree of emotional engagement is one or more of the individual being disengaged, disinterested, distracted, bored, day-dreaming, frustrated, nervous, stressed, and overwhelmed. 
     
     
         3 . The method of  claim 1 , wherein the knowledges arises from one or more of the human observer, the observer processor, the individual processor and the one or more remote processors. 
     
     
         4 . The method of  claim 1 , wherein the human interaction entity comprises one or more of a cartoon-like character, an avatar, a synthesized person, a synthesized pet, and an image that is familiar to the individual. 
     
     
         5 . The method of  claim 1 , further comprising transmitting one or more of the knowledge, a time the knowledge was expressed, and the observed degree of emotional engagement from one or more of the individual processor, the observer processor and the one or more remote processors to one or more monitoring processors configured to communicate with one or more monitoring humans other than the human observer and the individual. 
     
     
         6 . The method of  claim 1 , wherein the individual comprises a one of a child and a student. 
     
     
         7 . The method of  claim 1 , wherein the human observer comprises one of a parent, a guardian, a grandparent, a sibling, a teacher, a professor, a work colleague, an online assistant, and a tutor. 
     
     
         8 . The method of  claim 1 , wherein the individual output device comprises one or more of a computer monitor, a smart screen, a laptop, a mobile device, a phone, a tablet, a smart watch, an augmented reality headwear, a virtual reality headwear, a mixed reality headwear, a holographic projector, and a light-field projector. 
     
     
         9 . The method of  claim 1 , wherein an artificial intelligence is instantiated within one or more of the individual processor, the observer processor, and the one or more remote processors. 
     
     
         10 . The method of  claim 9 , wherein the artificial intelligence includes an artificial intelligence personality familiar to one or both of the individual and the human observer. 
     
     
         11 . A method for a human observer of an individual to effect a target degree of emotional engagement by the individual, comprising:
 providing, in proximity to the individual, an individual electronic device that includes an individual processor, one or more individual output devices operatively coupled to the individual processor, and one or more individual sensors operatively coupled to the individual processor;   providing, in proximity to the human observer, an observer electronic device that includes an observer processor, one or more observer output devices operatively coupled to the observer processor, and one or more observer sensors operatively coupled to the observer processor;   presenting knowledge to the individual that may be unknown to the individual on the one or more individual output devices;   acquiring, from the one or more individual sensors, interaction data from an interaction between the individual and the individual electronic device related to the knowledge that was presented;   presenting the interaction data on the one or more observer output devices;   indicating, by the human observer using the one or more observer sensors, an activity intended to direct an observed degree of emotional engagement by the individual toward a target degree of emotional engagement; and   presenting a human interaction entity on the one or more individual output devices to initiate the activity with the individual.   
     
     
         12 . The method of  claim 11 , wherein the individual degree of emotional engagement is one or more of the individual being disengaged, disinterested, distracted, bored, day-dreaming, frustrated, nervous, stressed, and overwhelmed. 
     
     
         13 . The method of  claim 11 , wherein the knowledges arises from one or more of the human observer, the observer processor, the individual processor and the one or more remote processors. 
     
     
         14 . The method of  claim 11 , wherein the human interaction entity comprises one or more of a cartoon-like character, an avatar, a synthesized person, a synthesized pet, and an image that is familiar to the individual. 
     
     
         15 . A method for a human observer of an individual to effect a target degree of emotional engagement by the individual, comprising:
 providing, in proximity to the individual, an individual electronic device that includes an individual processor, one or more individual output devices operatively coupled to the individual processor, and one or more individual sensors operatively coupled to the individual processor;   providing, in proximity to the human observer, an observer electronic device that includes an observer processor, one or more observer output devices operatively coupled to the observer processor, and one or more observer sensors operatively coupled to the observer processor;   presenting knowledge to the individual that may be unknown to the individual on the one or more individual output devices;   acquiring, from the one or more individual sensors, interaction data from an interaction between the individual and the individual electronic device related to the knowledge that was presented;   classifying within the interaction data, by one or more of the individual processor, the observer processor and one or more remote processors, an individual degree of emotional engagement by the individual in response to the knowledge that was presented,   presenting the individual degree of emotional engagement on the one or more observer output devices;   indicating, by the human observer using the one or more observer sensors, an activity intended to direct the individual degree of emotional engagement toward a target degree of emotional engagement; and   presenting a human interaction entity on the one or more individual output devices to initiate the activity with the individual.   
     
     
         16 . The method of  claim 15 , wherein the individual degree of emotional engagement is one or more of the individual being disengaged, disinterested, distracted, bored, day-dreaming, frustrated, nervous, stressed, and overwhelmed. 
     
     
         17 . The method of  claim 15 , wherein the knowledges arises from one or more of the human observer, the observer processor, the individual processor and the one or more remote processors. 
     
     
         18 . The method of  claim 15 , wherein the human interaction entity comprises one or more of a cartoon-like character, an avatar, a synthesized person, a synthesized pet, and an image that is familiar to the individual. 
     
     
         19 . The method of  claim 15 , wherein classifying the one or more indications of the individual degree of emotional engagement comprises identifying within the interaction data one or more of one or more words, phrases, interjections, utterances, gestures, smiles, hand movements, configurations of hand digits, written symbols, head movements, facial muscle movements, movements of forehead muscles, eye movements, changes in pupil size, eyelid blinks, electronic device button pushes, electronic device screen touches, shakes of the individual electronic device, movements of one or more objects sensed by the electronic device, eye gaze directions, changes in voice tone, changes in voice volume, changes in rates of speaking, changes in reaction times, lack of reaction to comments, lack of reaction to questions, and reactions unrelated to questions. 
     
     
         20 . The method of  claim 15 , further comprising transmitting one or more of the knowledge, a time the knowledge was expressed, and the individual degree of emotional engagement from one or more of the individual processor, the observer processor and the one or more remote processors to one or more monitoring processors configured to communicate with one or more monitoring humans other than the human observer and the individual.