Systems and Methods for Assessing Viewer Engagement

Abstract

A system for quantifying viewer engagement with a video playing on a display includes at least one camera to acquire image data of a viewing area in front of the display. A microphone acquires audio data emitted by a speaker coupled to the display. The system also includes a memory to store processor-executable instructions and a processor. Upon execution of the processor-executable instructions, the processor receives the image data and the audio data and determines an identity of the video displayed on the display based on the audio data. The processor also estimates a first number of people present in the viewing area and a second number of people engaged with the video. The processor further quantifies the viewer engagement of the video based on the first number of people and the second number of people.

1 . A method of quantifying viewer engagement with a video shown on a display, the method comprising:
 acquiring, with at least one camera, images of a viewing area in front of the display while the video is being shown on the display;   acquiring, with a microphone, audio data representing a soundtrack of the video emitted by a speaker coupled to the display;   determining, with a processor operably coupled to the at least one camera and the processor, an identity of the video based at least in part on the audio data;   estimating, with the processor and based at least in part on the image data, a first number of people present in the viewing area while the video is being shown on the display and a second number of people engaged with the video in the viewing area; and   transmitting, by the processor, the identity of the video, the first number of people, and the second number of people to a remote server.   
     
     
         2 . The method of  claim 1 , wherein acquiring the images comprises acquiring a first image of the viewing area using a visible camera and acquiring a second image of the viewing area using an infrared (IR) camera. 
     
     
         3 . The method of  claim 2 , wherein estimating the first number of people in the viewing area comprises:
 estimating a first raw number of people from the first image data and a second raw number of people from the second image data; and   comparing the first raw number with the second raw number to detect possible error in at least one of the first raw number or the second raw number.   
     
     
         4 . The method of  claim 1 , wherein acquiring the image data comprises acquiring images of the viewing area at a frame rate substantially equal to or greater than 20 frames per second. 
     
     
         5 . The method of  claim 1 , wherein acquiring the audio data comprises acquiring the audio data at an acquisition rate of about 0.1 Hz. 
     
     
         6 . The method of  claim 1 , wherein determining the identity of the video is based on audio signal fingerprinting. 
     
     
         7 . The method of  claim 1 , wherein estimating the first number of people present in the viewing area is based on body skeleton detection. 
     
     
         8 . The method of  claim 1 , wherein estimating the second number of people engaged with the at least on video is based on eye tracking. 
     
     
         9 . The method of  claim 1 , further comprising:
 quantifying the viewer engagement of the video based at least in part on the first number of people and the second number of people at each house household in the plurality of households.   
     
     
         10 . The method of  claim 9 , wherein quantifying the viewer engagement comprises:
 estimating an attention rate for the video, the attention rate representing a ratio of the second number of people engaged with the video to the first number of people in the viewing area; and   for each unique video in the plurality of videos, determining an attention index based on the attention rates of the videos in the plurality of videos.   
     
     
         11 . The method of  claim 10 , wherein the video is a unique video in a plurality of videos and the method further comprises:
 estimating a viewer count and a positive duration ratio based on the image data and on demographic information about each household in the plurality of households, the viewer count representing the second number of people engaged with each unique video and the positive duration ratio representing a ratio of total time spent by people in the plurality of households watching the unique video to a duration of the unique video.   
     
     
         12 . The method of  claim 9 , further comprising:
 determining an identity of each person present in the viewing area based at least in part on the image data,   wherein quantifying the viewer engagement of the video comprises quantifying the viewer engagement for each identified person.   
     
     
         13 . The method of  claim 9 , further comprising:
 transmitting the first number of people and the second number of people to a remote server, wherein quantifying the viewer engagement is carried out at the remote server.   
     
     
         14 . The method of  claim 9 , further comprising:
 determining whether a predetermined video in the plurality of videos is being displayed on the display based at least in part on the audio data, wherein quantifying the viewer engagement is based at least in part on whether the predetermined video is being displayed.   
     
     
         15 . The method of  claim 1 , further comprising:
 storing the first number of people and the second number of people in a memory operably coupled to the processor; and   erasing and/or overwriting the image data.   
     
     
         16 . The method of  claim 1 , further comprising:
 estimating an emotion of each person present in the viewing area.   
     
     
         17 . The method of  claim 1 , further comprising:
 estimating demographic information for each person in the viewing area from the image data.   
     
     
         18 . The method of  claim 17 , wherein estimating the demographic information comprises estimating age, gender, ethnicity group, and facial expression. 
     
     
         19 . A method of quantifying viewer engagement for unique videos in a plurality of videos, the method comprising:
 at each household in a plurality of households, acquiring image data of a viewing area in front of a display;   determining if the display is showing a video in the plurality of videos;   for each unique video in the plurality of videos, estimating (i) a viewing rate and (ii) a watching rate based on the image data and on demographic information about each household in the plurality of households, the viewing rate representing a ratio of a total number of people in the viewing areas to a total number of displays showing videos and the watching rate representing a ratio of a total number of people in households with display showing videos to a total number of people in the plurality of households; and   for each unique video in the plurality of videos, determining a viewability index based on the viewing rate and the watching rate.   
     
     
         20 . The method of  claim 19 , further comprising:
 for each unique video in the plurality of videos, estimating (iii) a viewer count and (iv) a positive duration ratio based on the image data and on demographic information about each household in the plurality of households, the viewer count representing a total number of people engaged with each unique video and the positive duration ratio representing a ratio of total time spent by people in the plurality of households watching the unique video to a duration of the unique video; and   weighting the viewability index based on the viewer count and the positive duration ratio.   
     
     
         21 . The method of  claim 20 , further comprising:
 normalizing the viewability index across the unique videos in the plurality of videos.   
     
     
         22 . The method of  claim 19 , wherein acquiring the image data comprises acquiring a first image of the viewing area using an optical camera and acquiring a second image of the viewing area using an infrared (IR) camera. 
     
     
         23 . The method of  claim 19 , wherein determining if the display is showing the video is based at least in part on audio data of the viewing area via signal fingerprinting technique. 
     
     
         24 . The method of  claim 19 , further comprising:
 transmitting the viewing rate and the watching rate to a remote server, wherein the viewability index is estimated by the remote server.   
     
     
         25 . A system for quantifying viewer engagement with a video playing on a display, the system comprising:
 at least one camera, disposed to image a viewing area in front of the display, to acquire image data of the viewing area;   a microphone, disposed in proximity to the display, to acquire audio data representing a soundtrack of the video emitted by a speaker coupled to the display;   a memory, operably coupled to the at least one camera and the microphone, to store processor-executable instructions; and   a processor, operably coupled to the at least one camera, the microphone, and the memory, wherein upon execution of the processor-executable instructions, the processor:
 determines an identity of the video based at least in part on the audio data; 
 estimates, based at least in part on the image data, a first number of people present in the viewing area while the video is being shown on the display and a second number of people engaged with the video in the viewing area; and 
 transmits the identity of the video, the first number of people, and the second number of people to a remote server. 
   
     
     
         26 . The system of  claim 25 , wherein the video comprises a television program provided via a set-top box and the processor is not connected to the set-top box. 
     
     
         27 . The system of  claim 25 , wherein the at least one camera comprises a visible camera and an infrared camera and the image data comprises a first image acquired by the visible camera and a second image acquired by the infrared camera. 
     
     
         28 . The system of  claim 27 , wherein upon execution of the processor-executable instructions, the processor further:
 estimates a first raw number of people from the first image and a second raw number of people from the second image; and   compares the first raw number with the second raw number to detect possible error in at least one of the first raw number or the second raw number.   
     
     
         29 . The system of  claim 25 , wherein upon execution of the processor-executable instructions, the processor:
 stores the first number of people and the second number of people in the memory; and   erases and/or overwrites the image data.   
     
     
         30 . The system of  claim 25 , further comprising:
 a network interface, operably coupled to the processor, to transmit the first number of people and the second number of people to a remote server.