Methods for training emotional response predictors utilizing attention in visual objects

Abstract

Described herein are methods for training a predictor of a user's emotional response to stimuli (e.g., digital content). In order to more accurately learn the nature of the emotional response of the user to the stimuli, in some embodiments, the training of the predictor involves collection of attention level data that indicates to which objects the user paid attention. The attention level data may be utilized to weight token instances representing visual objects from the stimuli. Such a weighting can help to train the emotional response predictor to better determine which objects influence the user's affective response and/or the extent of their influence on the user's affective response. In different embodiments, attention level information may come from different sources, such as eye tracking data of the user, and a model for predicting an interest level of the user in various visual objects.

What is claimed is: 
     
         1 . A method for training an emotional response predictor, comprising:
 receiving temporal windows of token instances to which a user was exposed; wherein each temporal window of token instances comprises at least two token instances that have overlapping instantiation periods, with each of the at least two token instances representing a different visual object;   receiving, for each temporal window of token instances, corresponding attention levels indicative of an extent of attention of the user in each of the different visual objects represented by the at least two token instances comprised in the temporal window of token instances;   generating samples corresponding to the temporal windows of token instances; wherein for each temporal window of token instances, values in its corresponding sample are indicative of weights of the token instances belonging to the temporal window of token instances; and wherein values in the sample that correspond to the at least two token instances of the temporal window of token instances, are further weighted according to their corresponding attention levels;   receiving target values corresponding to the temporal windows of token instances; wherein each target value corresponding to the temporal window of token instances represents an affective response of the user to being exposed to the token instances in the temporal window of token instances; and   training the emotional response predictor utilizing a machine learning-based training algorithm that uses the samples and the target values as training data.   
     
     
         2 . The method of  claim 1 , further comprising receiving eye tracking data of the user, which is indicative of objects the user gazed at while viewing content represented by a temporal window of token instances, and determining attention levels in at least two token instances from the temporal window of token instances; wherein the at least two token instances have overlapping instantiation periods, and each of the at least two token instances represents a different visual object. 
     
     
         3 . The method of  claim 1 , further comprising determining attention in at least two token instances from the temporal window of token instances utilizing a model for predicting an interest level of the user in various visual objects; wherein the at least two token instances have overlapping instantiation periods, and each of the at least two token instances represents a different visual object. 
     
     
         4 . The method of  claim 1 , further comprising determining attention in at least two token instances from the temporal window of token instances utilizing analysis of previous observations of interest of the user in token instances; wherein the at least two token instances have overlapping instantiation periods, and each of the at least two token instances represents a different visual object. 
     
     
         5 . The method of  claim 1 , further comprising receiving eye tracking data of other users indicative of objects the other users gazed at while viewing content represented by a temporal window of token instances, and determining attention levels in at least two token instances from the temporal window of token instances; wherein the at least two token instances have overlapping instantiation periods, and each of the at least two token instances represents a different visual object. 
     
     
         6 . The method of  claim 1 , further comprising using the emotional response predictor for predicting emotional state of the user after being exposed to a new temporal window of token instances; wherein the emotional response predictor utilizes a sample derived from the new temporal window of token instances in which values corresponding to the at least two token instances of the new temporal window of token instances are weighted according to attention levels of the user in the at least two token instances. 
     
     
         7 . The method of  claim 1 , wherein training the emotional response predictor involves training at least two different machine learning-based emotional response predictors on data collected over periods where the user was in different situations. 
     
     
         8 . The method of  claim 1 , wherein the training involves at least one of the following actions: setting parameters of a regression model utilized by the emotional response predictor to make its predictions, and setting weights of a neural network utilized by the emotional response predictor to make its predictions, setting parameters of a support vector machine for regression utilized by the emotional response predictor to make its predictions. 
     
     
         9 . The method of  claim 1 , further comprising receiving additional samples comprising temporal windows of token instances that do not have corresponding target values, and training the emotional response predictor utilizing a semi-supervised training method. 
     
     
         10 . The method of  claim 1 , further comprising determining a target value representing an affective response of the user to being exposed to content represented by a temporal window of token instances utilizing a model trained to predict an emotional state based on values obtained from a user measurement channel of the user. 
     
     
         11 . The method of  claim 1 , further comprising obtaining a value of a user measurement channel of the user by measuring the user with a sensor while the user is exposed to a temporal window of token instances, and utilizing the value of the user measurement channel of the user to determine a target value corresponding to the temporal window of token instances. 
     
     
         12 . A method for training an emotional response predictor that utilizes eye tracking, comprising:
 receiving temporal windows of token instances to which a user was exposed; wherein each temporal window of token instances comprises at least two token instances that have overlapping instantiation periods, with each of the at least two token instances representing a different visual object;   receiving, for each temporal window of token instances, eye tracking data of the user, which is indicative of objects the user gazed at while viewing content represented by the temporal window of token instances, and determining attention levels in at the least two token instances from the temporal window of token instances that have overlapping instantiation periods and represent different visual objects.   generating samples corresponding to the temporal windows of token instances; wherein for each temporal window of token instances, values in its corresponding sample are indicative of weights of the token instances belonging to the temporal window of token instances; and wherein values in the sample that correspond to the at least two token instances of the temporal window of token instances, are further weighted according to their corresponding attention levels;   receiving target values corresponding to the temporal windows of token instances; wherein each target value corresponding to the temporal window of token instances represents an affective response of the user to being exposed to the token instances in the temporal window of token instances; and   training the emotional response predictor utilizing a machine learning-based training algorithm that uses the samples and the target values as training data.   
     
     
         13 . The method of  claim 12 , further comprising using the emotional response predictor for predicting emotional state of the user after being exposed to a new temporal window of token instances; wherein the emotional response predictor utilizes a sample derived from the new temporal window of token instances in which values corresponding to the at least two token instances of the new temporal window of token instances are weighted according to attention levels of the user in the at least two token instances. 
     
     
         14 . The method of  claim 12 , wherein the training involves at least one of the following actions: setting parameters of a regression model utilized by the emotional response predictor to make its predictions, and setting weights of a neural network utilized by the emotional response predictor to make its predictions, setting parameters of a support vector machine for regression utilized by the emotional response predictor to make its predictions. 
     
     
         15 . The method of  claim 12 , further comprising determining a target value representing an affective response of the user to being exposed to content represented by a temporal window of token instances utilizing a model trained to predict an emotional state based on values obtained from a user measurement channel of the user. 
     
     
         16 . The method of  claim 12 , further comprising obtaining a value of a user measurement channel of the user by measuring the user with a sensor while the user is exposed to a temporal window of token instances, and utilizing the value of the user measurement channel of the user to determine a target value corresponding to the temporal window of token instances. 
     
     
         17 . A method for training an emotional response predictor for a user by utilizing eye tracking data of other users, comprising:
 receiving temporal windows of token instances to which a user was exposed; wherein each temporal window of token instances comprises at least two token instances that have overlapping instantiation periods, with each of the at least two token instances representing a different visual object;   receiving, for each temporal window of token instances, eye tracking data of the other users, which is indicative of objects the other users gazed at while viewing content represented by the temporal window of token instances, and determining attention levels in at the least two token instances from the temporal window of token instances that have overlapping instantiation periods and represent different visual objects.   generating samples corresponding to the temporal windows of token instances; wherein for each temporal window of token instances, values in its corresponding sample are indicative of weights of the token instances belonging to the temporal window of token instances; and wherein values in the sample that correspond to the at least two token instances of the temporal window of token instances, are further weighted according to their corresponding attention levels;   receiving target values corresponding to the temporal windows of token instances; wherein each target value corresponding to the temporal window of token instances represents an affective response of the user to being exposed to the token instances in the temporal window of token instances; and   training the emotional response predictor utilizing a machine learning-based training algorithm that uses the samples and the target values as training data.   
     
     
         18 . The method of  claim 17 , further comprising using the emotional response predictor for predicting emotional state of the user after being exposed to a new temporal window of token instances; wherein the emotional response predictor utilizes a sample derived from the new temporal window of token instances in which values corresponding to the at least two token instances of the new temporal window of token instances are weighted according to attention levels of the user in the at least two token instances. 
     
     
         19 . The method of  claim 17 , wherein the training involves at least one of the following actions: setting parameters of a regression model utilized by the emotional response predictor to make its predictions, and setting weights of a neural network utilized by the emotional response predictor to make its predictions, setting parameters of a support vector machine for regression utilized by the emotional response predictor to make its predictions. 
     
     
         20 . The method of  claim 17 , further comprising determining a target value representing an affective response of the user to being exposed to content represented by a temporal window of token instances utilizing a model trained to predict an emotional state based on values obtained from a user measurement channel of the user.