Live video latency measurement and control

Abstract

Technologies for measuring and controlling live video latency are disclosed. Embodiments capture a live video scene, ingest the live video scene into a live video stream, and encode the live video stream with data that can be used to compute latency measurements. Embodiments communicate the live video stream to a content distribution network. The live video stream is distributed, directly or indirectly by the content distribution network, to one or more user systems. The one or more user systems present the live video stream to one or more users. Embodiments determine a latency of the live video stream based on, for example, a measurement that is obtained during the capturing of the live video stream and another measurement that is obtained during or in response to the presenting of the live video stream to the one or more users.

What is claimed is: 
     
       1. A method, comprising:
 capturing a live video scene with a first device; 
 ingesting the live video scene into a video segment; 
 determining a first arrival time of the video segment at an ingestion server; 
 encoding the video segment with a first timestamp associated with the first arrival time; 
 creating a producer reference time box (PRFT) for the video segment; 
 distributing the video segment to one or more second devices, wherein the one or more second devices present the video segment to one or more users; 
 simulating receiving of the video segment at the one or more second devices to compute an estimated render time; 
 determining a latency of the video segment based on a plurality of measurements of time between a first time prior to the video segment being presented by the one or more second devices to the one or more users and a second time at or after the video segment is presented by the one or more second devices to the one or more users, wherein the plurality of measurements of time comprise an elapsed time between a time indicated by the PRFT and at least two of: completion of the encoding of the video segment, serving of the video segment by one or more servers of a content distribution network, or the estimated render time; and 
 based on the plurality of measurements, identifying a particular component of a live video transmission pipeline as contributing to increased latency in the distribution of the live video scene from the first device to the one or more second devices. 
 
     
     
       2. The method of  claim 1 , wherein determining a latency of the video segment comprises:
 extracting, from the video segment, the first timestamp associated with the first arrival time; 
 determining a second arrival time of the video segment at the one or more second devices; and 
 computing the latency of the video segment as a combination of the first arrival time and the second arrival time. 
 
     
     
       3. The method of  claim 2 , wherein setting a target latency is based on an engagement level associated with the video segment. 
     
     
       4. The method of  claim 2  further comprising:
 manipulating a header of the video segment to include the first arrival time, the second arrival time, and the first timestamp. 
 
     
     
       5. The method of  claim 4  further comprising:
 generating a pipeline offset based on the pipeline latency; and 
 manipulating the header of the video segment to include the pipeline offset. 
 
     
     
       6. The method of  claim 1  further comprising:
 determining that the latency of the video segment is greater than a threshold latency for a subset of the one or more second devices; and 
 notifying the users of the subset of the one or more second devices that the latency of the video segment is greater than the threshold latency. 
 
     
     
       7. The method of  claim 6  further comprising:
 determining that adjusting a type, a connection, or an application of the subset of the one or more second devices causes in a reduction in the latency of the video segment; and 
 notifying the users of the subset of the one or more second devices that an adjustment of the type, the connection, or the application reduces the latency of the video segment. 
 
     
     
       8. The method of  claim 1  further comprising controlling a networking path from the first device to the one or more second devices based on the latency. 
     
     
       9. The method of  claim 1 , further comprising:
 monitoring feedback from the one or more second devices to the first device, wherein the feedback comprises an interaction of the one or more users in response to the video segment; and 
 setting a target latency for the video segment based on the feedback. 
 
     
     
       10. The method of  claim 1 , wherein simulating the receiving of the video segment comprises:
 simulating performance of a plurality of different types of user systems; and 
 estimating a render time for a combination of a simulated user system of a particular type, a video presentation application executed by the simulated user system, and a network connection of the simulated user system. 
 
     
     
       11. A system comprising:
 a memory component; and 
 a processing device, coupled to the memory component, configured to perform operations comprising: 
 creating a producer reference time box (PRFT) for a video segment; 
 distributing the video segment to one or more second devices, wherein the one or more second devices present the video segment to one or more users; 
 simulating receiving of the video segment at the one or more second devices to compute an estimated render time; 
 determining a latency of the video segment based on a plurality of measurements of time between a first time prior to the video segment being presented by the one or more second devices to the one or more users and a second time at or after the video segment is presented by the one or more second devices to the one or more users, wherein the plurality of measurements of time comprise an elapsed time between a time indicated by the PRFT and at least two of: completion of the encoding of the video segment, serving of the video segment by one or more servers of a content distribution network, or the estimated render time, wherein determining the latency of the video segment comprises determining a pipeline latency that indicates latency measured between an ingestion server and an origin server of a live video transmission pipeline, and computing a render latency for the video segment that indicates an amount of time between a second arrival time of the video segment at a second device and a time that the video segment is presented by the second device; and 
 based on the plurality of measurements, identifying a particular component of the live video transmission pipeline as contributing to increased latency in the distribution of a live video scene from the first device to the one or more second devices. 
 
     
     
       12. The system of  claim 11 , wherein the operation of determining a latency of the video segment comprises:
 determining a first arrival time of the video segment at an ingestion server; 
 determining a second arrival time of the video segment at the one or more second devices; 
 generating a pipeline offset based on a first timestamp corresponding to the first time and a second timestamp corresponding to the second time; and 
 computing the latency of the video segment as a combination of the first arrival time, the second arrival time, and the pipeline offset. 
 
     
     
       13. The system of  claim 9 , wherein the latency is inversely related to the feedback. 
     
     
       14. The system of  claim 12 , the operations further comprising:
 manipulating a header of the video segment to include the first arrival time, the second arrival time, the first timestamp, or the second timestamp. 
 
     
     
       15. The system of  claim 11 , the operations further comprising:
 determining that the latency of the video segment is greater than a threshold latency for a subset of the one or more second devices; and 
 notifying the users of the subset of the one or more second devices that the latency of the video segment is greater than the threshold latency. 
 
     
     
       16. The system of  claim 15 , the operations further comprising:
 determining that adjusting a type, a connection, or an application of the subset of the one or more second devices causes in a reduction in the latency of the video segment; and 
 notifying the users of the subset of the one or more second devices that an adjustment of the type, the connection, or the application reduces the latency of the video segment. 
 
     
     
       17. The system of  claim 11 , the operations further comprising controlling a networking path from the first device to the one or more second devices based on the latency. 
     
     
       18. The system of  claim 11 , the operations further comprising:
 monitoring feedback from the one or more second devices to the first device, wherein the feedback comprises an interaction of the one or more users in response to the video segment; and 
 setting a target latency for the video segment based on the feedback. 
 
     
     
       19. A method of controlling a latency of a video stream, the method comprising:
 capturing a live video scene with a first device; 
 ingesting the live video scene into a plurality of video segments; 
 determining a first arrival time of the live video scene at an ingestion server; 
 generating a first timestamp associated with the first arrival time for each video segment of the plurality of video segments; 
 encoding the plurality of video segments, wherein the encoding comprises inserting the first timestamp into a header of each of the video segments of the plurality of video segments; 
 creating a producer reference time box (PRFT) for each of the video segments of the plurality of video segments; 
 distributing the plurality of video segments to a second device that presents the video segment to one or more users; 
 simulating receiving of the video segment at the one or more second devices to compute an estimated render time; 
 determining a latency of each video segment of the plurality of video segments based on a plurality of measurements of time comprising an elapsed time between a time indicated by the PRFT and at least two of: completion of the encoding of the video segment, serving of the video segment by one or more servers of a content distribution network, or the estimated render time, wherein determining the latency of the video segment comprises determining a pipeline latency that indicates latency measured between an ingestion server and an origin server of a live video transmission pipeline, and computing a render latency for the video segment that indicates an amount of time between a second arrival time of the video segment at a second device and a time that the video segment is presented by the second device; and 
 based on the plurality of measurements, identifying a particular component of the live video transmission pipeline as contributing to increased latency in the distribution of the live video scene from the first device to the one or more second devices. 
 
     
     
       20. The method of  claim 19 , wherein determining a latency of each video segment comprises:
 generating a second timestamp associated with a second arrival time for each video segment of the plurality of video segments at the second device; 
 extracting, from the header of each video segment, the first timestamp; and 
 computing the latency of the video segment as a combination of the first timestamp and the second timestamp. 
 
     
     
       21. The method of  claim 20  further comprising computing the render latency for each video segment of the plurality of video segments. 
     
     
       22. The method of  claim 19 , further comprising:
 monitoring feedback from the one or more second devices to the first device, wherein the feedback comprises an interaction of the one or more users in response to the video segment; and 
 setting a target latency for the video segment based on the feedback.