Coordination and increased utilization of graphics processors during inference

Abstract

A mechanism is described for facilitating inference coordination and processing utilization for machine learning at autonomous machines. A method of embodiments, as described herein, includes detecting, at training time, information relating to one or more tasks to be performed according to a training dataset relating to a processor including a graphics processor. The method may further include analyzing the information to determine one or more portions of hardware relating to the processor capable of supporting the one or more tasks, and configuring the hardware to pre-select the one or more portions to perform the one or more tasks, while other portions of the hardware remain available for other tasks.

What is claimed is: 
     
       1. A method comprising:
 scheduling shared resources in a system for a plurality of contexts associated with a plurality of clients of the system, wherein the shared resources include one or more graphics processors; 
 receiving specification of a limitation on usage of the one or more graphics processors by one or more contexts of the plurality of contexts; and 
 upon determining that the limitation on usage of the one or more graphics processors is specified for the one or more contexts:
 limiting usage of the one or more graphics processors by the one or more contexts, wherein limiting usage of the one or more graphics processors includes assigning threads for each of the one or more contexts to a specified portion of available threads of the one or more graphics processors, the specified portion being less than all available threads of the one or more graphics processors, limiting execution of the threads for the one or more contexts to the specified portion of available threads assigned to each of the one or more contexts; and 
 reserving execution threads for use by remaining contexts of the plurality of contexts. 
 
 
     
     
       2. The method of  claim 1 , further comprising limiting the one or more contexts to a subset of the one or more graphics processors, the subset being less than all available graphics processors. 
     
     
       3. The method of  claim 2 , wherein usage of the one or more graphics processors by one or more contexts is limited in part to increase utilization of the one or more graphics processors. 
     
     
       4. The method of  claim 3 , further comprising monitoring utilization of the one or more graphics processors. 
     
     
       5. The method of  claim 3 , additionally comprising adjusting a limitation on threads for the one or more contexts based in part on a control target and a demand from a scheduler. 
     
     
       6. The method of  claim 1 , wherein the one or more graphics processors include a single instruction multiple thread (SIMT) architecture. 
     
     
       7. The method of  claim 6 , wherein the SIMT architecture includes hardware multithreading. 
     
     
       8. A system comprising:
 a memory device; and 
 one or more processors including one or more graphics processors, the one or more processors to execute instructions stored on the memory device, wherein the instructions cause the one or more processors to perform operations comprising: 
 performing processing for one or more contexts of a client of the system, the system configured to execute a plurality of contexts associated with a plurality of clients; and 
 accessing scheduled resources of the system for the one or more contexts, the resources being shared by the plurality of clients, wherein shared resources of the system include the one or more graphics processors; 
 wherein access to the one or more graphics processors by the one or more contexts is subject to a limitation on usage set for one or more contexts of the plurality of contexts; 
 wherein the limitation on usage includes a limitation on threads of the one or more graphics processors for the one or more contexts to a specified portion of available threads, the specified portion being less than all available threads of the one or more graphics processors; and 
 wherein the limitation on usage reserves threads for use by the plurality of contexts other than the one or more contexts for which the limitation on usage is set. 
 
     
     
       9. The system of  claim 8 , wherein the specified portion of available threads for the limitation on usage of the one or more graphics processors is specified by the system and the limitation on usage limits the one or more contexts to a subset of the one or more graphics processors, the subset being less than all available graphics processors. 
     
     
       10. The system of  claim 9 , wherein the limitation on threads for the one or more contexts is provided at least in part to increase utilization of the one or more graphics processors. 
     
     
       11. The system of  claim 10 , wherein utilization of the one or more graphics processors is monitored by the system. 
     
     
       12. The system of  claim 10 , the operations additionally comprising adjusting the limitation on threads for the one or more contexts based in part on a control target and a demand from a scheduler. 
     
     
       13. The system of  claim 8 , wherein the one or more graphics processors include a single instruction multiple thread (SIMT) architecture. 
     
     
       14. The system of  claim 13 , wherein the SIMT architecture includes hardware multithreading. 
     
     
       15. An apparatus comprising:
 a processing system including one or more processors, the one or more processors including one or more graphics processors, each of the one or more graphics processors including a plurality of processing resources; 
 a scheduler to receive specification of a limitation on usage of the plurality of processing resources by the plurality of contexts and schedule shared resources in the processing system for a plurality of contexts associated with a plurality of clients of the processing system according to the limitation on usage; and 
 wherein the processing system has a capability to limit usage of the plurality of processing resources of the one or more graphics processors by the plurality of contexts based on the specification of the limitation on usage, wherein the limitation on usage of the plurality of processing resources of the one or more graphics processors includes to limit execution of threads of each context of the plurality of contexts to a specified portion of available threads provided by the plurality of processing resources of the one or more graphics processors, the specified portion being less than all available threads provided by the plurality of processing resources of the one or more graphics processors. 
 
     
     
       16. The apparatus of  claim 15 , wherein the processing system is to specify the portion of available threads for the limitation on usage of the one or more graphics processors and the limitation on usage is to limit one or more of the plurality of contexts to a subset of the one or more graphics processors, the subset being less than all available graphics processors. 
     
     
       17. The apparatus of  claim 16 , wherein the limitation on threads for the one or more contexts is provided by the processing system at least in part to increase utilization of the one or more graphics processors. 
     
     
       18. The apparatus of  claim 17 , wherein the processing system is further to monitor utilization of the one or more graphics processors. 
     
     
       19. The apparatus of  claim 15 , wherein the one or more graphics processors include a single instruction multiple thread (SIMT) architecture. 
     
     
       20. The apparatus of  claim 19 , wherein the SIMT architecture includes hardware multithreading.