Class-based intelligent multiplexing over unmanaged networks

Abstract

An electronic device sends a content stream, via an unmanaged network, toward a client device and monitors the capacity of the unmanaged network. The device determines whether an aggregate bandwidth of an upcoming portion of the content stream fits the capacity. The upcoming portion of the content stream includes video content and user-interface data. In response to a determination that the aggregate bandwidth of the upcoming portion of the content stream does not fit the capacity, when the user-interface data is not the result of a user interaction: the device prioritizes a frame rate of the video content over latency for the user-interface data, and in accordance with a determination that the aggregate bandwidth of the upcoming portion of the content stream does not fit the capacity, sends ahead one or more frames of the video content in the upcoming portion, and delays the user-interface data in the upcoming portion.

What is claimed is: 
     
       1. A method of adapting content-stream bandwidth, comprising:
 sending a content stream, via an unmanaged network, toward a client device; 
 monitoring a capacity of the unmanaged network; 
 determining whether an aggregate bandwidth of an upcoming portion of the content stream fits the capacity, wherein the upcoming portion of the content stream includes video content and user-interface data; 
 in response to a determination that the aggregate bandwidth of the upcoming portion of the content stream does not fit the capacity: 
 when the user-interface data is not the result of a user interaction:
 prioritizing a frame rate of the video content over latency for the user-interface data; and 
 in accordance with a determination that the aggregate bandwidth of the upcoming portion of the content stream does not fit the capacity: 
 sending ahead one or more frames of the video content in the upcoming portion, and
 delaying the user-interface data in the upcoming portion. 
 
 
 
     
     
       2. The method of  claim 1 , wherein the upcoming portion of the content stream corresponds to a respective frame time. 
     
     
       3. The method of  claim 1 , wherein:
 the upcoming portion of the content stream further includes audio data; and 
 the method further comprises, when the user interface data is not the result of a user interaction, sending the audio data ahead in a portion of the content stream that precedes the upcoming portion. 
 
     
     
       4. The method of  claim 1 , wherein delaying the user-interface data comprises spreading the user-interface data over the upcoming portion and a second portion of the content stream that follows the upcoming portion. 
     
     
       5. The method of  claim 1 , wherein:
 the upcoming portion of the content stream further includes audio data; and 
 the method further comprises, when the user interface data is not the result of a user interaction, leaving the audio data unaffected. 
 
     
     
       6. The method of  claim 1 , wherein:
 sending the content stream comprises sending bursts of TCP segments; and 
 monitoring the capacity of the unmanaged network comprises receiving acknowledgments of the bursts, the acknowledgments including timestamps, and using the timestamps to determine the capacity of the unmanaged network. 
 
     
     
       7. The method of  claim 6 , wherein using the timestamps to determine the capacity of the unmanaged network comprises:
 building a histogram of arrival times in accordance with the timestamps; and 
 deriving the capacity from the histogram. 
 
     
     
       8. The method of  claim 1 , further comprising:
 in response to determining that decreasing the frame rate of the video content does not sufficiently reduce the aggregate bandwidth of the upcoming portion of the content stream, decreasing a frame rate of the user-interface data; 
 decreasing the frame rate of the video content comprises decreasing the frame rate to a half, third, or quarter of the frame rate of the content stream. 
 
     
     
       9. An electronic device, comprising:
 one or more processors; and 
 memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for: 
 sending a content stream, via an unmanaged network, toward a client device; 
 monitoring a capacity of the unmanaged network; 
 determining whether an aggregate bandwidth of an upcoming portion of the content stream fits the capacity, wherein the upcoming portion of the content stream includes video content and user-interface data; 
 in response to a determination that the aggregate bandwidth of the upcoming portion of the content stream does not fit the capacity: 
 when the user-interface data is not the result of a user interaction:
 prioritizing a frame rate of the video content over latency for the user-interface data; and 
 in accordance with a determination that the aggregate bandwidth of the upcoming portion of the content stream does not fit the capacity: 
 sending ahead one or more frames of the video content in the upcoming portion, and 
 delaying the user-interface data in the upcoming portion. 
 
 
     
     
       10. The electronic device of  claim 9 , wherein the upcoming portion of the content stream corresponds to a respective frame time. 
     
     
       11. The electronic device of  claim 9 , wherein:
 the upcoming portion of the content stream further includes audio data; and 
 the method further comprises, when the user interface data is not the result of a user interaction, sending the audio data ahead in a portion of the content stream that precedes the upcoming portion. 
 
     
     
       12. The electronic device of  claim 9 , wherein delaying the user-interface data comprises spreading the user-interface data over the upcoming portion and a second portion of the content stream that follows the upcoming portion. 
     
     
       13. The electronic device of  claim 9 , wherein:
 the upcoming portion of the content stream further includes audio data; and 
 the method further comprises, when the user interface data is not the result of a user interaction, leaving the audio data unaffected. 
 
     
     
       14. The electronic device of  claim 9 , wherein:
 sending the content stream comprises sending bursts of TCP segments; and 
 monitoring the capacity of the unmanaged network comprises receiving acknowledgments of the bursts, the acknowledgments including timestamps, and using the timestamps to determine the capacity of the unmanaged network. 
 
     
     
       15. The electronic device of  claim 14 , wherein using the timestamps to determine the capacity of the unmanaged network comprises:
 building a histogram of arrival times in accordance with the timestamps; and 
 deriving the capacity from the histogram. 
 
     
     
       16. The electronic device of  claim 9 , further comprising:
 in response to determining that decreasing the frame rate of the video content does not sufficiently reduce the aggregate bandwidth of the upcoming portion of the content stream, decreasing a frame rate of the user-interface data; 
 decreasing the frame rate of the video content comprises decreasing the frame rate to a half, third, or quarter of the frame rate of the content stream. 
 
     
     
       17. A non-transitory computer-readable storage medium storing one or more programs configured for execution by an electronic device, the one or more programs comprising instructions for:
 sending a content stream, via an unmanaged network, toward a client device; 
 monitoring a capacity of the unmanaged network; 
 determining whether an aggregate bandwidth of an upcoming portion of the content stream fits the capacity, wherein the upcoming portion of the content stream includes video content and user-interface data; 
 in response to a determination that the aggregate bandwidth of the upcoming portion of the content stream does not fit the capacity: 
 when the user-interface data is not the result of a user interaction:
 prioritizing a frame rate of the video content over latency for the user-interface data; and 
 in accordance with a determination that the aggregate bandwidth of the upcoming portion of the content stream does not fit the capacity: 
 sending ahead one or more frames of the video content in the upcoming portion, and 
 delaying the user-interface data in the upcoming portion. 
 
 
     
     
       18. The storage medium of  claim 17 , wherein the upcoming portion of the content stream corresponds to a respective frame time. 
     
     
       19. The storage medium of  claim 17 , wherein:
 the upcoming portion of the content stream further includes audio data; and 
 the method further comprises, when the user interface data is not the result of a user interaction, sending the audio data ahead in a portion of the content stream that precedes the upcoming portion. 
 
     
     
       20. The storage medium of  claim 17 , wherein delaying the user-interface data comprises spreading the user-interface data over the upcoming portion and a second portion of the content stream that follows the upcoming portion. 
     
     
       21. The storage medium of  claim 17 , wherein:
 the upcoming portion of the content stream further includes audio data; and 
 the method further comprises, when the user interface data is not the result of a user interaction, leaving the audio data unaffected. 
 
     
     
       22. The storage medium of  claim 17 , wherein:
 sending the content stream comprises sending bursts of TCP segments; and 
 monitoring the capacity of the unmanaged network comprises receiving acknowledgments of the bursts, the acknowledgments including timestamps, and using the timestamps to determine the capacity of the unmanaged network. 
 
     
     
       23. The storage medium of  claim 22 , wherein using the timestamps to determine the capacity of the unmanaged network comprises:
 building a histogram of arrival times in accordance with the timestamps; and 
 deriving the capacity from the histogram. 
 
     
     
       24. The storage medium of  claim 17 , further comprising:
 in response to determining that decreasing the frame rate of the video content does not sufficiently reduce the aggregate bandwidth of the upcoming portion of the content stream, decreasing a frame rate of the user-interface data; 
 decreasing the frame rate of the video content comprises decreasing the frame rate to a half, third, or quarter of the frame rate of the content stream.