Robot navigation for personal assistance

Abstract

Techniques for performing robotic assistance are disclosed. A plurality of images of an individual is obtained by an imagery module associated with an autonomous mobile robot. Cognitive state data including facial data for the individual in the plurality of images is identified by an analysis module associated with the autonomous mobile robot. A facial expression metric, based on the facial data for the individual in the plurality of images, is calculated. A cognitive state metric for the individual is generated by the analysis module based on the cognitive state data. The autonomous mobile robot initiates one or more responses based on the cognitive state metric. The one or more responses include one or more electromechanical responses. The one or more electromechanical responses cause the robot to change locations.

What is claimed is: 
     
         1 . A computer-implemented method for robotic assistance comprising:
 obtaining, by an imagery module associated with an autonomous mobile robot, a plurality of images of an individual;   identifying, by an analysis module associated with the autonomous mobile robot, cognitive state data including facial data for the individual in the plurality of images;   calculating a facial expression metric based on the facial data for the individual;   generating, by the analysis module, a cognitive state metric for the individual based on the facial expression metric; and   causing the autonomous mobile robot to initiate one or more responses, wherein the one or more responses are based on the cognitive state metric.   
     
     
         2 . The method of  claim 1  wherein the one or more responses include one or more electromechanical responses. 
     
     
         3 . The method of  claim 2  wherein the one or more electromechanical responses cause the robot to change locations. 
     
     
         4 . The method of  claim 2  wherein the one or more electromechanical responses are based on a distance from the robot to the individual. 
     
     
         5 . The method of  claim 1  further comprising causing the autonomous mobile robot to initiate one or more image changes on a display on the robot, wherein the one or more image changes are based on the cognitive state metric. 
     
     
         6 . The method of  claim 1  wherein the identifying uses one or more classifiers. 
     
     
         7 . The method of  claim 6  wherein the one or more classifiers are generated using machine learning. 
     
     
         8 . (canceled) 
     
     
         9 . The method of  claim 1  wherein the one or more responses are based on valence for the individual as determined from the cognitive state metric. 
     
     
         10 . The method of  claim 1  further comprising determining context within which the robot is operating. 
     
     
         11 . The method of  claim 1  further comprising determining context for the individual. 
     
     
         12 . The method of  claim 1  wherein the identifying includes facial expression analysis. 
     
     
         13 . The method of  claim 1  further comprising generating a second cognitive state metric. 
     
     
         14 . The method of  claim 13  further comprising causing the autonomous mobile robot to initiate one or more further responses based on the second cognitive state metric. 
     
     
         15 . The method of  claim 14  wherein the one or more further responses include electromechanical responses. 
     
     
         16 . The method of  claim 13  further comprising identifying a difference in cognitive states for the individual. 
     
     
         17 . The method of  claim 16  wherein causing the autonomous mobile robot to initiate one or more electromechanical responses is based on the second cognitive state metric and the difference in cognitive states. 
     
     
         18 . The method of  claim 16  wherein the difference occurs over a time duration. 
     
     
         19 . The method of  claim 16  wherein the difference occurs between the individual and a second individual. 
     
     
         20 . The method of  claim 1  wherein the one or more responses produce motion in the robot. 
     
     
         21 . The method of  claim 1  wherein the robot includes a social robot. 
     
     
         22 . The method of  claim 1  further comprising causing the autonomous mobile robot to provide information to the individual, wherein the information is based on the cognitive state. 
     
     
         23 . The method of  claim 1  further comprising causing the autonomous mobile robot to provide information to a third party, wherein the information is based on the cognitive state. 
     
     
         24 . The method of  claim 1  further comprising:
 identifying, by the analysis module, multiple human faces in the plurality of images; 
 defining, by the analysis module, a region of interest (ROI) in one of the plurality of images for two or more identified human faces; 
 extracting, by the analysis module, one or more histogram-of-oriented-gradients (HoG) features from each ROI; and 
 computing, by the analysis module, a set of facial metrics based on the one or more HoG features for each of the multiple human faces. 
 
     
     
         25 . The method of  claim 1  wherein the generating is further based on speech information from the individual. 
     
     
         26 - 27 . (canceled) 
     
     
         28 . A computer program product embodied in a non-transitory computer readable medium for robotic assistance comprising code which causes one or more processors to perform operations of:
 obtaining, by an imagery module associated with an autonomous mobile robot, a plurality of images of an individual;   identifying, by an analysis module associated with the autonomous mobile robot, cognitive state data including facial data for the individual in the plurality of images;   calculating a facial expression metric based on the facial data for the individual;   generating, by the analysis module, a cognitive state metric for the individual based on the facial expression metric; and   causing the autonomous mobile robot to initiate one or more responses, wherein the one or more responses are based on the cognitive state metric.   
     
     
         29 . A computer system for robotic assistance comprising:
 a memory which stores instructions;   one or more processors coupled to the memory wherein the one or more processors, when executing the instructions which are stored, are configured to:
 obtain, by an imagery module associated with an autonomous mobile robot, a plurality of images of an individual; 
 identify, by an analysis module associated with the autonomous mobile robot, cognitive state data including facial data for the individual in the plurality of images; 
 calculate a facial expression metric based on the facial data for the individual; 
 generate, by the analysis module, a cognitive state metric for the individual based on the facial expression metric; and 
 cause the autonomous mobile robot to initiate one or more responses, wherein the one or more responses are based on the cognitive state metric.