Analyzing viewer behavior in real time

Abstract

Systems, methods and articles of manufacture for are provided for analyzing user behavior in real time by ingesting telemetry data related to a streaming media application; feeding the telemetry data to a machine learning model (MLM) that produces a User Experience (UX) command based on the telemetry data and prior telemetry data received from the content streaming application; selecting content items to provide to the client device based on the telemetry data; determining, based on the telemetry data, whether the client device has sufficient free resources to receive the UX command and the content items in a current time window while providing a predefined level of service; when client device has sufficient free resources to receive the UX command and the content items, encapsulating the UX command with the content items in a content stream; and transmitting the content stream to the client device.

What is claimed is: 
     
       1. A method, comprising:
 ingesting, in real-time, current telemetry data from a client device related to a streaming media application; 
 feeding the current telemetry data to one or more machine learning models (MLMs) of a behavior analysis engine that produces, in real-time, one or more User Experience (UX) commands based on results obtained from feeding the current telemetry data and prior telemetry data received from the streaming media application into the one or more MLMs, the one or more MLMs including an anomaly detector trained to recommend adjustments based on identifying how network characteristics of client devices affect content delivery, wherein the one or more UX commands comprise a delivery command to adjust content delivery settings on the client device, and wherein feeding the current telemetry data to the one or more MLMs comprises identifying the content delivery settings based on operation data in the current telemetry data and in the prior telemetry data, the content delivery settings comprising content delivery pathway settings; 
 selecting content items to provide to the client device based on the current telemetry data; 
 in response to determining that the client device has sufficient free resources to receive the one or more UX commands in addition to the content items in a current transmission time window while providing a predefined level of service, encapsulating the one or more UX commands with the content items in a content stream; and 
 transmitting the content stream to the client device, wherein the content delivery pathway settings on the client device are adjusted based on the prior telemetry data, which indicate an optimized state of the content delivery pathway settings. 
 
     
     
       2. The method of  claim 1 , wherein feeding the current telemetry data to the one or more MLMs further comprises:
 identifying a new content item for provision to the client device based on playback data in the current telemetry data and in the prior telemetry data; and 
 identifying software settings in the streaming media application based on interaction data in the current telemetry data and in the prior telemetry data. 
 
     
     
       3. The method of  claim 1 , wherein feeding the current telemetry data to the one or more MLMs further comprises identifying a new content item for provision to the client device based on playback data in the current telemetry data and in the prior telemetry data,
 wherein the one or more UX commands further comprise a content command that specifies to cache the new content item on the client device. 
 
     
     
       4. The method of  claim 1 , wherein feeding the current telemetry data to the one or more MLMs further comprises identifying software settings in the streaming media application based on interaction data in the current telemetry data and in the prior telemetry data,
 wherein the one or more UX commands further comprise a User Interface (UI) command that specifies to adjust the software settings to optimize a UX on the client device based on the prior telemetry data, which indicate an optimized state of the software settings. 
 
     
     
       5. The method of  claim 1 , further comprising:
 in response to determining that the client device does not have sufficient free resources to receive the one or more UX commands in addition to the content items in the current transmission time window while providing the predefined level of service, encapsulating the content items in a second content stream without the one or more UX commands. 
 
     
     
       6. The method of  claim 1 , wherein the one or more MLMs include a long short-term memory unit, and wherein the current telemetry data fed into the one or more MLMs in the current transmission time window are treated as the prior telemetry data for subsequent telemetry data fed into the one or more MLMs in a subsequent time window. 
     
     
       7. The method of  claim 1 , wherein the one or more MLMs further include at least one of a recommender or a UX engine. 
     
     
       8. A system, comprising:
 a processor; and 
 a memory having instructions embodied therewith that when executed by the processor enable the system to perform an operation comprising: 
 ingesting, in real-time, current telemetry data from a client device related to a streaming media application; 
 feeding the current telemetry data to one or more machine learning models (MLMs) of a behavior analysis engine that produces, in real-time, one or more User Experience (UX) commands based on results obtained from feeding the current telemetry data and prior telemetry data received from the streaming media application into the one or more MLMs, the one or more MLMs including an anomaly detector trained to recommend adjustments based on identifying how network characteristics of client devices affect content delivery, wherein the one or more UX commands comprise a delivery command to adjust content delivery settings on the client device, and wherein feeding the current telemetry data to the one or more MLMs comprises identifying the content delivery settings based on operation data in the current telemetry data and in the prior telemetry data, the content delivery settings comprising content delivery pathway settings; 
 selecting content items to provide to the client device based on the current telemetry data; 
 in response to determining that the client device has sufficient free resources to receive the one or more UX commands in addition to the content items in a current transmission time window while providing a predefined level of service, encapsulating the one or more UX commands with the content items in a content stream; and 
 transmitting the content stream to the client device, wherein the content delivery pathway settings on the client device are adjusted based on the prior telemetry data, which indicate an optimized state of the content delivery pathway settings. 
 
     
     
       9. The system of  claim 8 , wherein feeding the current telemetry data to the one or more MLMs further comprises:
 identifying a new content item for provision to the client device based on playback data in the current telemetry data and in the prior telemetry data; and 
 identifying software settings in the streaming media application based on interaction data in the current telemetry data and in the prior telemetry data. 
 
     
     
       10. The system of  claim 8 , wherein feeding the current telemetry data to the one or more MLMs further comprises identifying a new content item for provision to the client device based on playback data in the current telemetry data and in the prior telemetry data, and
 wherein the one or more UX commands further comprise a content command that specifies to cache the new content item on the client device. 
 
     
     
       11. The system of  claim 8 , wherein feeding the current telemetry data to the one or more MLMs further comprises identifying software settings in the streaming media application based on interaction data in the current telemetry data and in the prior telemetry data, and
 wherein the one or more UX commands further comprise a User Interface (UI) command that specifies to adjust the software settings to optimize a UX on the client device based on the prior telemetry data, which indicate an optimized state of the software settings. 
 
     
     
       12. The system of  claim 8 , wherein the operation further com prises:
 in response to determining that the client device does not have sufficient free resources to receive the one or more UX commands in addition to the content items in the current transmission time window while providing the predefined level of service, encapsulating the content items in a second content stream without the one or more UX commands. 
 
     
     
       13. The system of  claim 8 , wherein the one or more MLMs include a long short-term memory unit, and wherein the current telemetry data fed into the one or more MLMs in the current transmission time window are treated as the prior telemetry data for subsequent telemetry data fed into the one or more MLMs in a subsequent time window. 
     
     
       14. The system of  claim 8 , wherein the one or more MLMs further include at least one of a recommender or a UX engine. 
     
     
       15. A system, comprising:
 a processor; 
 a memory having instructions embodied therewith that when executed by the processor enable the system to perform an operation comprising:
 gathering, in real-time, telemetry data for the system related to a streaming media application receiving a content stream; 
 
 in response to determining that a free bandwidth of bandwidth available to the system while receiving the content stream satisfies a threshold, transmitting, in real-time, the telemetry data to a streaming server providing the content stream, the streaming server including a behavior analysis engine configured to produce one or more User Experience (UX) commands based on results obtained from feeding the telemetry data to one or more machine learning models (MLMs), the one or more MLMs including an anomaly detector trained to recommend adjustments based on identifying how network characteristics of client devices affect content delivery, wherein the one or more UX commands comprise a delivery command, and wherein feeding the telemetry data to the one or more MLMs comprises identifying content delivery settings based on operation data in the telemetry data, the content delivery settings comprising content delivery pathway settings; and 
 in response to receiving the delivery command, adjusting the content delivery settings, including the content delivery pathway settings, in the streaming media application based on the telemetry data, which indicate an optimized state of the content delivery pathway settings, wherein the delivery command is included in the content stream. 
 
     
     
       16. The system of  claim 15 , wherein the telemetry data are aggregated on the system. 
     
     
       17. The system of  claim 15 , wherein the one or more UX commands further comprise a content command, and wherein the operation further comprises, in response to receiving the content command, adjusting, in the streaming media application, how much content to cache on the system. 
     
     
       18. The system of  claim 15 , wherein the one or more UX commands further comprise a User Interface (UI) command, and wherein the operation further comprises, in response to receiving the UI command, rearranging a layout of a UI of the streaming media application. 
     
     
       19. The system of  claim 15 , wherein adjusting the content delivery settings in the streaming media application includes at least one of:
 adjusting a format of the content stream; 
 adjusting a rate or quality of the content stream; 
 selecting a new intermediary source from which to request the content stream; or selecting a new host locale for the content stream. 
 
     
     
       20. The system of  claim 15 , wherein the telemetry data include:
 playback data related to playback of a content item included in the content stream; 
 interaction data related to actions made with respect to User Interface (UI) elements within the streaming media application; and 
 operational data related to network and system status. 
 
     
     
       21. The system of  claim 15 , wherein the one or more MLMs further include at least one of a recommender or a UX engine.