Rendering video frames for a user interface operation performed at a client device

Abstract

In some implementations, a device includes one or more processors and a non-transitory memory. In some implementations, a method includes obtaining a request for a sequence of video frames that corresponds to a user interface operation being performed at a client device. In some implementations, the sequence of video frames is to be presented at the client device at a first frame rate. In some implementations, the method includes determining an availability of computing resources associated with providing the sequence of video frames to the client device. In some implementations, the method includes generating, based on the availability of computing resources, the sequence of video frames at a second frame rate that is greater than the first frame rate. In some implementations, the method includes triggering the client device to present the sequence of video frames at the first frame rate.

What is claimed is: 
     
       1. A method comprising:
 at a first device including a non-transitory memory and one or more processors:
 obtaining a request for a sequence of video frames that corresponds to a user interface operation being performed at a client device that is different from the first device, wherein the sequence of video frames is to be presented at the client device at a first frame rate; 
 determining an availability of computing resources at the first device associated with generating the sequence of video frames for the client device; 
 in response to the availability of computing resources associated with generating the sequence of video frames being greater than a threshold amount of computing resources, generating the sequence of video frames at a second frame rate that is greater than the first frame rate; and 
 triggering the client device to present the sequence of video frames at the first frame rate. 
 
 
     
     
       2. The method of  claim 1 , wherein determining the availability of computing resources comprises determining a number of available processing units; and
 wherein generating the sequence of video frames at the second frame rate comprises generating the sequence of video frames at the second frame rate in response to the number of available processing units being greater than a threshold number of processing units. 
 
     
     
       3. The method of  claim 1 , wherein determining the availability of computing resources comprises determining an amount of available compute time; and
 wherein generating the sequence of video frames comprises generating the sequence of video frames at the second frame rate in response to the amount of available compute time being greater than a threshold amount of time. 
 
     
     
       4. The method of  claim 1 , wherein determining the availability of computing resources comprises determining a number of client devices that are expected to request respective sequences of video frames for corresponding user interface operations; and
 wherein generating the sequence of video frames comprises generating the sequence of video frames at the second frame rate in response to the number of client devices that are expected to request respective sequences of video frames being less than a threshold number of client devices. 
 
     
     
       5. The method of  claim 1 , wherein the availability of computing resources is a function of a time of day; and
 wherein generating the sequence of video frames comprises generating the sequence of video frames at the second frame rate in response to a current time being within a time period that corresponds to the second frame rate. 
 
     
     
       6. The method of  claim 1 , further comprising:
 providing, to an encoder, the sequence of video frames and an encoding guidance vector that provides encoding guidance to the encoder in order to allow the encoder to encode the sequence of video frames within a threshold amount of time. 
 
     
     
       7. The method of  claim 6 , wherein the encoding guidance vector indicates that there is no change between a first video frame and a second video frame that appear in succession in the sequence of video frames in order to allow the encoder to conserve encoding resources by forgoing analysis of the second video frame. 
     
     
       8. The method of  claim 7 , further comprising replacing the second video frame with a null frame. 
     
     
       9. The method of  claim 6 , wherein the encoding guidance vector indicates an area within which to detect changes between a first video frame and a second video frame that appear in succession in the sequence of video frames. 
     
     
       10. The method of  claim 6 , wherein the encoding guidance vector includes a motion vector that indicates a movement of a block between a first video frame and a second video frame that appear in succession in the sequence of video frames. 
     
     
       11. The method of  claim 1 , further comprising determining the first frame rate based on client resource availability data that indicates an amount of resources that are available to the client device. 
     
     
       12. The method of  claim 1 , further comprising determining the first frame rate based on an availability of computing resources associated with transmitting the sequence of video frames to the client device. 
     
     
       13. The method of  claim 1 , wherein the determining the availability of computing resources comprises determining an availability of graphics generation resources associated with generating the sequence of video frames. 
     
     
       14. The method of  claim 1 , wherein the determining the availability of computing resources comprises determining an availability of encoding resources associated with encoding the sequence of video frames. 
     
     
       15. The method of  claim 1 , wherein triggering the client device to present the sequence of video frames at the first frame rate comprises:
 transmitting the sequence of video frames to an edge device; and 
 instructing the edge device to transmit the sequence of video frames to the client device in a sequential manner at the first frame rate. 
 
     
     
       16. The method of  claim 1 , wherein triggering the client device to present the sequence of video frames at the first frame rate comprises:
 transmitting the sequence of video frames to the client device; and 
 instructing the client device to present the sequence of video frames at the first frame rate. 
 
     
     
       17. A first device comprising:
 one or more processors; 
 a non-transitory memory; and 
 one or more programs stored in the non-transitory memory, which, when executed by the one or more processors, cause the first device to:
 obtain a request for a sequence of video frames that corresponds to a user interface operation being performed at a client device that is different from the first device, wherein the sequence of video frames is to be presented at the client device at a first frame rate; 
 determine an availability of computing resources at the first device associated with generating the sequence of video frames for the client device; 
 in response to the availability of computing resources associated with generating the sequence of video frames being greater than a threshold amount of computing resources, generate the sequence of video frames at a second frame rate that is greater than the first frame rate; and 
 trigger the client device to present the sequence of video frames at the first frame rate. 
 
 
     
     
       18. The first device of  claim 17 , wherein determining the availability of computing resources comprises determining a number of available processing units; and
 wherein generating the sequence of video frames at the second frame rate comprises generating the sequence of video frames at the second frame rate in response to the number of available processing units being greater than a threshold number of processing units. 
 
     
     
       19. A non-transitory memory storing one or more programs, which, when executed by one or more processors of a first device, cause the first device to:
 obtain a request for a sequence of video frames that corresponds to a user interface operation being performed at a client device that is different from the first device, wherein the sequence of video frames is to be presented at the client device at a first frame rate; 
 determine an availability of computing resources at the first device associated with generating the sequence of video frames for the client device; 
 in response to the availability of computing resources associated with generating the sequence of video frames being greater than a threshold amount of computing resources, generate the sequence of video frames at a second frame rate that is greater than the first frame rate; and 
 trigger the client device to present the sequence of video frames at the first frame rate. 
 
     
     
       20. The non-transitory memory of  claim 19 , wherein determining the availability of computing resources comprises determining a number of client devices that are expected to request respective sequences of video frames for corresponding user interface operations; and
 wherein generating the sequence of video frames comprises generating the sequence of video frames at the second frame rate in response to the number of client devices that are expected to request respective sequences of video frames being less than a threshold number of client devices.