Prediction and use of processor inactivity for rendering frames

Abstract

A computing device performs a first operation before a first commit deadline, resulting in a first frame being rendered and displayed in a first cycle. A second operation is performed, before a second commit deadline, resulting in a second frame being rendered and displayed in a second cycle. A time remaining to a third commit deadline is determined, using the current time. A third operation is predicted, performable before a third commit deadline. An additional operation is predicted, performable for a future cycle. A total processing time for the third and additional operations is determined, being less than the remaining time. The third and additional operations are performed for use in a future cycle. The result of the third operation is used to render a third frame for the third cycle. The result of the additional operation is used to render an additional frame before a future render deadline.

What is claimed is: 
     
       1. A method comprising performing, by a computing device:
 performing a first operation before a first commit deadline, wherein a first result of the first operations is used for rendering a first frame; 
 displaying the first frame in a first cycle on a display of the computing device; 
 performing a second operation having a second result used for rendering a second frame for a second cycle, the second operation completed before a second commit deadline for the second cycle; 
 determining a remaining time to a third commit deadline relative to a current time; 
 predicting a third operation to be performed before the third commit deadline for a third cycle; 
 predicting an additional operation to be performed for cycles after the third cycle; 
 determining that a total processing time of the third operation and the additional operation is less than the remaining time to the third commit deadline; 
 responsive to determining that the total processing time is less than the remaining time, performing the third operation and the additional operation to be used in a future cycle; 
 rendering, using a third result of the third operation, a third frame for the third cycle; and 
 rendering, using an additional result of the additional operation, an additional frame for the future cycle before a future render deadline for the future cycle. 
 
     
     
       2. The method of  claim 1 , wherein the first commit deadline, the second commit deadline, and the third commit deadline are determined based on a refresh rate. 
     
     
       3. The method of  claim 1 , wherein the first commit deadline, the second commit deadline, and the third commit deadline have a variable periodicity. 
     
     
       4. The method of  claim 1 , wherein the additional operation is a prefetch operation. 
     
     
       5. The method of  claim 1 , wherein the second operation is responsive to a scrolling operation at a user interface of the computing device, and wherein the scrolling operation does not cause a portion of a new image to be displayed on the display of the computing device. 
     
     
       6. The method of  claim 5 , wherein the additional operation results in at least a portion of a new image to be displayed. 
     
     
       7. The method of  claim 1 , wherein the remaining time is determined by an update scheduler, and wherein the third operation and the additional operation are predicted using a prediction engine that predicts operations based on a current frame and an order of images to be displayed by an application. 
     
     
       8. The method of  claim 1 , wherein the additional operation is selected from among a plurality of operations that have a likelihood of future operation that is above a threshold. 
     
     
       9. A method comprising performing, by a computing device having commit deadlines with a variable periodicity:
 performing a first operation before a first commit deadline having a first period as measured between a previous commit deadline and the first commit deadline, wherein a first result of the first operations is used for rendering a first frame, wherein the first operation is implemented using one or more processing elements using a first power setting; 
 displaying the first frame in a first cycle on a display of the computing device; 
 identifying a second operation having a second result to be used for rendering a second frame for a second cycle, the second operation to be completed before a second commit deadline for the second cycle, the second commit deadline having a second period as measured from the first commit deadline to the second commit deadline, the second period different from the first period; 
 determining a second power setting of one or more processing elements such that the one or more processing elements complete the second operation before the second commit deadline, wherein the second power setting is lower than the first power setting; 
 performing the second operation using the one or more processing elements executing with the second power setting; and 
 rendering, using the second result of the second operation, the second frame for the second cycle. 
 
     
     
       10. The method of  claim 9 , wherein determining the second power setting includes:
 determining a remaining time to the second commit deadline relative to a current time; 
 determining a completion time using the first power setting; and 
 comparing the completion time to the remaining time to determine the second power setting. 
 
     
     
       11. The method of  claim 9 , wherein the first commit deadline and the second commit deadline are determined based on a refresh rate. 
     
     
       12. The method of  claim 9 , wherein determining the second power setting is performed by a scheduler that interfaces with a user interface routine, and wherein the scheduler instructs the one or more processing elements to use the second power setting. 
     
     
       13. The method of  claim 12 , wherein the scheduler estimates a completion time using the second power setting to be less than a remaining time before the second commit deadline. 
     
     
       14. The method of  claim 9 , wherein determining the second power setting is performed by a performance controller that receives information about commit deadlines and adjusts a power setting of the one or more processing elements. 
     
     
       15. An electronic device comprising:
 a memory storing a plurality of instructions; and 
 one or more processors communicably coupled with the memory, wherein the one or more processors are further configured to execute the plurality of instructions to perform a method comprising: 
 performing a first operation before a first commit deadline, wherein a first result of the first operations is used for rendering a first frame; 
 displaying the first frame in a first cycle on a display of the electronic device; 
 performing a second operation having a second result used for rendering a second frame for a second cycle, the second operation completed before a second commit deadline for the second cycle; 
 determining a remaining time to a third commit deadline relative to a current time; 
 predicting a third operation to be performed before the third commit deadline for a third cycle; 
 predicting an additional operation to be performed for cycles after the third cycle; 
 determining that a total processing time of the third operation and the additional operation is less than the remaining time to the third commit deadline; 
 responsive to determining that the total processing time is less than the remaining time, performing the third operation and the additional operation to be used in a future cycle; 
 rendering, using a third result of the third operation, a third frame for the third cycle; and 
 rendering, using an additional result of the additional operation, an additional frame for the future cycle before a future render deadline for the future cycle. 
 
     
     
       16. The electronic device of  claim 15 , wherein the first commit deadline, the second commit deadline, and the third commit deadline are determined based on a refresh rate. 
     
     
       17. The electronic device of  claim 15 , wherein the additional operation is a prefetch operation. 
     
     
       18. The electronic device of  claim 15 , wherein the second operation is responsive to a scrolling operation at a user interface of the electronic device, and wherein the scrolling operation does not cause a portion of a new image to be displayed on the display of the electronic device. 
     
     
       19. The electronic device of  claim 15 , wherein the remaining time is determined by an update scheduler, and wherein the third operation and the additional operation are predicted using a prediction engine that predicts operations based on a current frame and an order of images to be displayed by an application.