Data processing methods for predictions of media content performance

Abstract

Methods and systems of predicting performance data for a piece of media content that is consumable by a user at a client device are provided. In one or more embodiments, the method collects raw input data, such as from a webcam, indicative of a user's response to the media content as the user watches the content. The data is processed to extract and obtain a series of head pose signals and facial expression signals, which is then input to a classification model. The model maps the performance data of the media content over time in response to the signals evaluated by the method to produce a prediction of the performance of the piece of media content.

What is claimed is: 
     
         1 . A computer-implemented method of predicting performance data for a piece of media content that is consumable by a user at a client device, the method comprising:
 collecting, at the client device, raw input data indicative of a response of the user to the piece of media content during consumption of the piece of media content;   processing the collected raw input data to:
 extract a time series of descriptor data points, and 
 obtain a time series of emotional state data points; and 
   outputting predicted performance data for the piece of media content based on a classification model that maps between performance data and a predictive parameter of the time series of descriptor data points or the time series of emotional state data points,   wherein the predictive parameter is a quantitative indicator of relative change in the response of the user to the piece of media content.   
     
     
         2 . A computer-implemented method according to  claim 1 , wherein the piece of media content is consumable by a plurality of users, each of the plurality of users being at a respective client device, and wherein the method further comprises collecting, at each of a plurality of the respective client devices, raw input data indicative of a plurality of user responses to the piece of media content. 
     
     
         3 . A computer-implemented method according to  claim 1 , wherein the step of processing the collected raw input data further comprises:
 determining the predictive parameter;   applying a linear regression to the predictive parameter to output a linear regression output;   binarizing the linear regression output to produce a binarized output; and   applying a logistic regression to the binarized output to output the predicted performance data.   
     
     
         4 . A computer-implemented method according to  claim 2 , further comprising:
 calculating a temporal difference dx i   j  between the extracted time series of the descriptor data points or the time series of emotional state data points, wherein   
       
         
           
             
               
                 
                   dx 
                   i 
                   j 
                 
                 = 
                 
                   
                     dx 
                     j 
                   
                   
                     dt 
                     i 
                   
                 
               
               , 
             
           
         
       
       and wherein x j (t) is a time series of a quantitative parameter x of the extracted descriptor data point or the extracted emotional state data for a user j;
 normalizing the temporal difference to produce a normalized temporal difference dx i   j * by subtracting an average difference  dx j    from the temporal difference (dx i   j *=dx i   j − dx j   ), wherein 
 
       
         
           
             
               
                 
                   〈 
                   
                     dx 
                     j 
                   
                   〉 
                 
                 = 
                 
                   
                     
                       ∑ 
                       
                         i 
                         = 
                         0 
                       
                       T 
                     
                      
                     
                       dx 
                       i 
                       j 
                     
                   
                   T 
                 
               
               , 
             
           
         
       
       and T is a duration of the time series;
 segmenting the time series into a plurality of time bins having a predetermined duration; 
 calculating a maximum of the normalized differences according to mx k   j =max i∈k (dx i   j *), wherein the notation i∈k means that the ith value falls in bin k; 
 weighting and summing the values of each of the plurality of time bins according to Dx j =Σ k=1   n w k *mx k   j , wherein n is the number of bins so there is no more frame or segment index of the variable; 
 normalizing Dx j  by the length of the piece of media content; 
 generating a descriptive statistic indicative of the predictive parameter across the plurality of users. 
 
     
     
         5 . A computer-implemented method according to  claim 1 , wherein the client device is communicable with a server device over a network, and wherein the processing of the collected raw input data occurs at the server device. 
     
     
         6 . A computer-implemented method according to  claim 1 , wherein the raw input data includes any of user behavioral data, user physiological data or metadata relating to the piece of media content. 
     
     
         7 . A computer-implemented method according to  claim 1 , wherein each emotional state data point is determined based on one or more descriptor data points. 
     
     
         8 . A computer-implemented method according to  claim 7 , wherein each descriptor data point includes a quantitative parameter that is indicative of a feature extracted from the raw input data. 
     
     
         9 . A computer-implemented method according to  claim 8 , wherein the predictive parameter is a function of relative change of the quantitative parameter between adjacent emotional state data points in the time series of emotional state data points. 
     
     
         10 . A computer-implemented method according to  claim 1 , wherein each emotional state data point includes a quantitative parameter that is indicative of user emotional state. 
     
     
         11 . A computer-implemented method according to  claim 10 , wherein the predictive parameter is a function of relative change of the quantitative parameter between adjacent emotional state data points in the time series of emotional state data points. 
     
     
         12 . A computer-implemented method according to  claim 2  further comprising:
 determining an individual predictive parameter from the time series of descriptor data points or the time series of emotional state data points for each of the plurality of users; and 
 determining a group predictive parameter from the individual predictive parameters of the plurality of users, 
 wherein the predicted performance data is obtained using the group predictive parameter. 
 
     
     
         13 . A computer-implemented method according to  claim 12 , wherein processing the collected data includes inputting the group predictive parameter into a classification model that maps between the group predictive parameter and the performance data. 
     
     
         14 . A computer-implemented method according to  claim 13  further comprising obtaining a plurality of group predictive parameters, wherein the classification model maps between the plurality of group predictive parameters and the performance data. 
     
     
         15 . A computer-implemented method according to  claim 13 , wherein the predicted performance data output is generated using a result output from the classification model. 
     
     
         16 . A computer-implemented method according to  claim 1 , wherein the raw input data comprises image data captured at the client device. 
     
     
         17 . A computer-implemented method according to  claim 16 , wherein the image data includes a plurality of image frames showing facial images of a user. 
     
     
         18 . A computer-implemented method according to  claim 7 , wherein each descriptor data point is a facial feature descriptor data point that is a multi-dimensional data point, each component of the multi-dimensional data point being indicative of a respective facial landmark. 
     
     
         19 . A computer-implemented method according  claim 18 , wherein each facial feature descriptor data point is associated with a respective frame. 
     
     
         20 . A computer-implemented method according to  claim 1 , wherein the piece of media content is any of a live video stream, a video commercial, an audio commercial, a movie trailer, a movie, a web advertisement, an animated game, or an image.