Networked device with suggested response to incoming message

Abstract

Disclosed are systems, devices, and methods for providing a response to a message. In one aspect, a method includes receiving, by a first electronic device, the incoming message, outputting, by the first electronic device, data derived from the incoming message, determining a state of a user of the first electronic device in response to the outputting of the data derived from the incoming message, and generating, by the first electronic device, a response message indicating the state.

What is claimed is: 
     
         1 . A non-transitory computer readable storage medium comprising instructions that when executed configure hardware processing circuitry to perform operations for responding to an incoming message, the operations comprising:
 receiving, by a first electronic device, the incoming message;   outputting, by the first electronic device, data derived from the incoming message;   in response to the outputting capturing an image with a camera of the first electronic device or capturing audio data with a microphone of the first electronic device;   determining a state at the first electronic device based on the captured image or the captured audio; and   generating, by the first electronic device, a response message indicating the state.   
     
     
         2 . The non-transitory computer readable storage medium of  claim 1 , the operations further comprising:
 receiving audio data from a microphone of the first electronic device in response to the outputting; and   determining the state further based on the audio data.   
     
     
         3 . A method of responding to an incoming message, comprising:
 receiving, by a first electronic device, the incoming message;   outputting, by the first electronic device, data derived from the incoming message;   determining a state of a user of the first electronic device in response to the outputting of the data derived from the incoming message; and   generating, by the first electronic device, a response message indicating the state.   
     
     
         4 . The method of  claim 3 , further comprising:
 capturing an image with a camera of the first electronic device; and   determining the state of the user based on the captured image.   
     
     
         5 . The method of  claim 4 , further comprising:
 determining a facial expression of the user based on the captured image; and   determining the state of the user based on the facial expression.   
     
     
         6 . The method of  claim 5 , further comprising:
 extracting facial features from the captured image;   generating a facial feature vector based on the extracted facial features;   generating a product vector based on a product of the facial feature vector and a facial expression matrix, wherein the state of the user is further determined based on the product vector.   
     
     
         7 . The method of  claim 3 , further comprising:
 receiving audio data from a microphone of the first electronic device; and   determining the state of the user based on the audio data.   
     
     
         8 . The method of  claim 7 , further comprising:
 extracting vocal features from the audio data;   generating a vocal feature vector based on the extracted vocal features;   generating a product vector based on a product of the vocal feature vector and a vocal expression matrix, wherein the state of the user is further determined based on the product vector.   
     
     
         9 . The method of  claim 3 , wherein the incoming message is received from a second device, the method further comprising receiving one or more additional messages from the second device, wherein the determining of the emotional state is further based on the one or more additional messages. 
     
     
         10 . The method of  claim 3 , wherein the outputting of the data derived from the message includes one or more of playing a video or an audio indicated by the incoming message on the first electronic device, or displaying an emoji or text indicated by the incoming message on an electronic display of the first electronic device. 
     
     
         11 . The method of  claim 3 , further comprising generating the response message to include one or more of a video clip, an image, an emoji, or text indicating the determined state. 
     
     
         12 . A system, comprising:
 hardware processing circuitry;   one or more hardware memories storing instructions that when executed configure the hardware processing circuitry to perform operations comprising:
 receiving, by a first electronic device, an incoming message; 
 outputting, by the first electronic device, data derived from the incoming message; 
 determining a state of a user of the first electronic device in response to the outputting of the data derived from the incoming message; and 
 generating a response message indicating the state; 
   
     
     
         13 . The system of  claim 12 , the operations further comprising:
 capturing an image with a camera of the first electronic device; and   determining the state of the user based on the captured image.   
     
     
         14 . The system of  claim 13 , the operations further comprising:
 determining a facial expression of the user based on the captured image; and   determining the state of the user based on the facial expression.   
     
     
         15 . The system of  claim 14 , the operations further comprising:
 extracting facial features from the captured image;   generating a facial feature vector based on the extracted facial features;   generating a product vector based on a product of the facial feature vector and a facial expression matrix, wherein the state of the user is further determined based on the product vector.   
     
     
         16 . The system of  claim 12 , the operations further comprising:
 receiving audio data from a microphone of the first electronic device; and   determining the state of the user based on the audio data.   
     
     
         17 . The system of  claim 16 , the operations further comprising:
 extracting vocal features from the audio data;   generating a vocal feature vector based on the extracted vocal features;   generating a product vector based on a product of the vocal feature vector and a vocal expression matrix, wherein the state of the user is further determined based on the product vector.   
     
     
         18 . The system of  claim 12 , wherein the incoming message is received from a second device, the method further comprising receiving one or more additional messages from the second device, wherein the determining of the emotional state is further based on the one or more additional messages. 
     
     
         19 . The system of  claim 12 , wherein the outputting of the data derived from the message includes one or more of playing a video or an audio indicated by the incoming message on the first electronic device, or displaying an emoji or text indicated by the incoming message on an electronic display of the first electronic device. 
     
     
         20 . The system of  claim 12 , the operations further comprising generating the response message to include one or more of a video, an image, an emoji, or text indicating the determined state.