Congestion control for low-latency interactive video streaming

Abstract

A server transmits an encoded game frame over a network to a respective client system as a set of packets. In response to transmitting the set of packets, the server determines a bandwidth estimate based on the size of the encoded game frame and the timing data associated with the transmitted set of packets. The server then compares the bandwidth estimate to a current video bitrate of the game stream being transmitted from the server to the respective client device. In response to the comparison indicating an underutilization of the network, the server increases the encoding bitrate. Further, in response to the comparison indication an overutilization of the network, the server decreases the encoding bitrate.

What is claimed is: 
     
         1 . A computer-implemented method for cloud-based gaming, comprising:
 in response to an encoder encoding a game frame of a stream of game frames according to an encoding bitrate to produce an encoded game frame, transmitting the encoded game frame as a set of packets over a network;   determining a bandwidth estimate based on timing data associated with the set of packets transmitted over the network; and   modifying the encoding bitrate based on the bandwidth estimate.   
     
     
         2 . The computer-implemented method of  claim 1 , wherein the timing data comprises a set of transmit time stamps and a set of receive time stamps associated with the transmitted set of packets. 
     
     
         3 . The computer-implemented method of  claim 1 , wherein modifying the encoding bitrate comprises:
 in response to the bandwidth estimate being less than a current video bitrate, decreasing the encoding bitrate.   
     
     
         4 . The computer-implemented method of  claim 1 , wherein modifying the encoding bitrate comprises:
 in response to the bandwidth estimate being greater than a current video bitrate, increasing the encoding bitrate.   
     
     
         5 . The computer-implemented method of  claim 1 , wherein determining the bandwidth estimate further comprises:
 determining interval data representing a time interval to transmit the transmitted encoded game frame based on the timing data;   and determining the bandwidth estimate based on the interval data.   
     
     
         6 . The computer-implemented method of  claim 1 , wherein determining the bandwidth estimate further comprises:
 smoothing the bandwidth estimate based on a step size.   
     
     
         7 . The computer-implemented method of  claim 1 , further comprising:
 determining a pacing rate based on the bandwidth estimate;   and in response to the encoder encoding a second game frame of the stream of game frames to produce a second encoded game frame, transmitting the second encoded game frame as a second set of packets over the network according to the pacing rate.   
     
     
         8 . The computer-implemented method of  claim 1 , wherein transmitting the encoded game frame comprises:
 sending the set of packets to a client system associated with a current client gaming session over the network; and   receiving a set of receive time stamps associated with the set of packets from the client system.   
     
     
         9 . The computer-implemented method of  claim 8 , further comprising:
 decoding, at the client system, the encoded game frame to produce a decoded game frame, wherein the decoded game frame represents at least a portion of a virtual environment associated with the current client gaming session.   
     
     
         10 . The method of  claim 8 , further comprising:
 rendering the game frame based on a gaming application associated with the current client gaming session.   
     
     
         11 . A system for cloud-based gaming, comprising:
 an encoder configured to encode a game frame of a stream of game frames according to an encoding bitrate to produce an encoded game frame;   a transport layer configured to transmit the encoded game frame as a set of packets over a network; and   a congestion control engine configured to:
 determine a bandwidth estimate based on timing data associated with the set of packets transmitted over the network; and 
 modify the encoding bitrate of the encoder based on the bandwidth estimate. 
   
     
     
         12 . The system of  claim 11 , wherein the timing data comprises a set of transmit time stamps and a set of receive time stamps associated with the transmitted set of packets. 
     
     
         13 . The system of  claim 11 , wherein the congestion control engine is further configured to:
 in response to the bandwidth estimate being lower than a current video bitrate, decrease the encoding bitrate of the encoder.   
     
     
         14 . The system of  claim 11 , wherein the congestion control engine is further configured to:
 in response to the bandwidth estimate being greater than a current video bitrate, increase the encoding bitrate of the encoder.   
     
     
         15 . The system of  claim 11 , wherein the congestion control engine is further configured to:
 determine interval data representing a time interval to transmit the transmitted encoded game frame based on the timing data;   and determine the bandwidth estimate based on the interval data.   
     
     
         16 . The system of  claim 11 , wherein the congestion control engine is further configured to:
 smooth the bandwidth estimate based on a step size.   
     
     
         17 . The system of  claim 11 , wherein:
 the congestion control engine is further configured to determine a pacing rate based on the bandwidth estimate, and   the transport layer is further configured to, in response to the encoder encoding a second game frame of the stream of game frames to produce a second encoded game frame, transmit the second encoded game frame as a second set of packets over the network according to the pacing rate.   
     
     
         18 . The system of  claim 11 , further comprising:
 a client system associated with a current client gaming session, wherein the transport layer is further configured to:
 send the set of packets to the client system; and 
 receive a set of receive time stamps associated with the set of packets from the client system. 
   
     
     
         19 . The system of  claim 18 , wherein the client system is configured to:
 decode the encoded game frame to produce a decoded game frame, wherein the decoded game frame represents at least a portion of a virtual environment associated with the current client gaming session.   
     
     
         20 . A system for cloud-based gaming, comprising:
 a cloud-based gaming server configured to render a set of game frames based on a current client gaming session, the cloud-based gaming server comprising:
 an encoder configured to encode a game frame of the set of game frames of a stream of game frames according to an encoding bitrate to produce an encoded game frame; 
 a transport layer configured to transmit the encoded game frame as a set of packets over a network to a respective client system based on the current client gaming session; and 
 a congestion control engine configured to:
 determine a bandwidth estimate based on timing data associated with the set of packets transmitted over the network to the respective client system; and 
 modify the encoding bitrate of the encoder based on the bandwidth estimate.