Interference-aware scheduling service for virtual GPU enabled systems

Abstract

Disclosed are aspects of interference-aware virtual machine assignment for systems that include graphics processing units (GPUs) that are virtual GPU (vGPU) enabled. In some examples, an interference function is used to predict interference for assignment of a workload to a graphics processing unit (GPU). The interference function outputs a predicted interference to place the workload on the GPU. The workload is assigned to the GPU based on a comparison of the predicted interference to a plurality of predicted interferences for the workload on various GPUs.

Therefore, the following is claimed: 
     
       1. A system comprising:
 at least one computing device comprising at least one processor; and; 
 at least one data store comprising machine readable instructions, wherein the instructions, when executed by the at least one processor, cause the at least one computing device to at least:
 identify kernel length types for kernels of a respective one of at least one workload, wherein a respective kernel is identified as a short kernel or a long kernel based at least in part on a comparison of an execution time of the respective kernel to a time slice; 
 provide, by a scheduling service executed by the at least one computing device, parameters comprising at least one baseline parameter for the respective one of the at least one workload currently assigned to a particular graphics processing unit (GPU) into an interference model that predicts interference between a particular workload and the at least one workload of the particular GPU in a computing environment comprising a plurality of GPUs, wherein the particular GPU uses the time slice for context switching, and the at least one baseline parameter comprises at least one parameter that is based on the kernel length types for the kernels; identify, by the scheduling service, an output from the interference model, the output comprising a predicted interference corresponding to placement of the particular workload on the particular GPU; and 
 assign, by the scheduling service, the particular workload to the particular GPU based on the predicted interference corresponding to a minimum predicted interference among a plurality of predicted interferences for at least a subset of the plurality of GPUs. 
 
 
     
     
       2. The system of  claim 1 , wherein the at least one baseline parameter comprises at least one of: an average kernel length, an average long kernel length indicating an average of long kernels that execute for longer than the time slice, and a ratio between a first number of short kernels that execute within the time slice of the particular GPU and a second number of the long kernels that execute for longer than the time slice length. 
     
     
       3. The system of  claim 2 , wherein the at least one baseline parameter further comprises at least one of: a GPU utilization, a peripheral component interconnect express (PCIe) read bandwidth, a PCIe write bandwidth, a virtual central processing unit (vCPU) utilization, and a workload memory utilization for the at least one workload. 
     
     
       4. The system of  claim 1 , wherein the machine readable instructions, when executed by the at least one processor, cause the at least one computing device to at least:
 identify a plurality of available GPUs, wherein the subset of the plurality of GPUs correspond to the plurality of available GPUs; and 
 determine the plurality of predicted interferences comprising a respective predicted interference for a respective one of the plurality of available GPUs. 
 
     
     
       5. The system of  claim 4 , wherein the machine readable instructions, when executed by the at least one processor, cause the at least one computing device to at least:
 input at least one baseline parameter for the respective one of the plurality of available GPUs to determine the respective predicted interference. 
 
     
     
       6. The system of  claim 5 , wherein the parameters input into the interference model further comprise at least one baseline parameter for the particular workload. 
     
     
       7. The system of  claim 1 , wherein the interference model identifies a one-to-one interference between two workloads, and the predicted interference identified as an output from the interference model corresponds to a worst-case predicted interference among at least one interference calculated for the at least one workload currently assigned to the particular GPU. 
     
     
       8. A method comprising:
 identifying kernel length types for kernels of a respective one of at least one workload, wherein a respective kernel is identified as a short kernel or a long kernel based at least in part on a comparison of an execution time of the respective kernel to a time slice; 
 providing, by a scheduling service executed by at least one computing device, parameters comprising at least one baseline parameter for the respective one of the at least one workload currently assigned to a particular graphics processing unit (GPU) into an interference model that predicts interference between a particular workload and the at least one workload of the particular GPU in a computing environment comprising a plurality of GPUs, wherein the particular GPU uses the time slice for context switching; 
 identifying, by the scheduling service, an output from the interference model, the output comprising a predicted interference to place a workload on the particular GPU; and 
 assigning, by the scheduling service, the workload to the particular GPU based on the predicted interference corresponding to a minimum predicted interference among a plurality of predicted interferences for at least a subset of the plurality of GPUs. 
 
     
     
       9. The method of  claim 8 , wherein the at least one baseline parameter comprises at least one of: an average kernel length, an average long kernel length indicating an average of long kernels that execute for longer than the time slice, and a ratio between a first number of short kernels that execute within the time slice length of the particular GPU and a second number of the long kernels that execute for longer than the time slice length. 
     
     
       10. The method of  claim 9 , wherein the at least one baseline parameter further comprises at least one of: a GPU utilization, a peripheral component interconnect express (PCIe) read bandwidth, a PCIe write bandwidth, a virtual central processing unit (vCPU) utilization, and a workload memory utilization for the at least one workload. 
     
     
       11. The method of  claim 8 , further comprising:
 training a plurality of interference models to predict interference based on measured interferences and a respective set of the at least one baseline parameter corresponding to a respective one of a plurality of workloads comprising the workload. 
 
     
     
       12. The method of  claim 8 , further comprising:
 determining that the interference model comprises a minimum error of a plurality of errors for a plurality of interference models; and 
 selecting the interference model to process the workload. 
 
     
     
       13. The method of  claim 12 , wherein the minimum error is a minimum average error, a minimum median error, or a minimum mode error. 
     
     
       14. The method of  claim 8 , wherein the at least the subset of the plurality of GPUs corresponds to a set of available GPUs for the workload within the computing environment. 
     
     
       15. A non-transitory computer-readable medium comprising machine readable instructions, wherein the instructions, when executed by at least one processor, cause at least one computing device to at least:
 identify kernel length types for kernels of a respective one of at least one workload, wherein a respective kernel is identified as a short kernel or a long kernel based at least in part on a comparison of an execution time of the respective kernel to a time slice; 
 provide, by a scheduling service executed by the at least one computing device, parameters comprising at least one baseline parameter for the respective one of the at least one workload currently assigned to a particular graphics processing unit (GPU) into an interference model that predicts interference between a particular workload and the at least one workload of the particular GPU in a computing environment comprising a plurality of GPUs, wherein the particular GPU uses the time slice for context switching; 
 identify, by the scheduling service, an output from an interference function, the output comprising a predicted interference to place a workload on the particular GPU; and 
 assign, by the scheduling service, the workload to the particular GPU based on a comparison of the predicted interference to a plurality of predicted interferences for at least a subset of the plurality of GPUs. 
 
     
     
       16. The non-transitory computer-readable medium of  claim 15 , wherein the at least one baseline parameter comprises at least one of: an average kernel length, an average long kernel length indicating an average of long kernels that execute for longer than the time slice, and a ratio between a first number of short kernels that execute within the time slice length of the particular GPU and a second number of the long kernels that execute for longer than the time slice length. 
     
     
       17. The non-transitory computer-readable medium of  claim 16 , wherein the at least one baseline parameter further comprises at least one of: a GPU utilization, a peripheral component interconnect express (PCIe) read bandwidth, a PCIe write bandwidth, a virtual central processing unit (vCPU) utilization, and a workload memory utilization for the at least one workload. 
     
     
       18. The non-transitory computer-readable medium of  claim 15 , wherein the machine readable instructions, when executed by the at least one processor, cause the at least one computing device to at least:
 identify a plurality of available GPUs, wherein the subset of the plurality of GPUs correspond to the plurality of available GPUs; and 
 determine the plurality of predicted interferences comprising a respective predicted interference for a respective one of the plurality of available GPUs. 
 
     
     
       19. The non-transitory computer-readable medium of  claim 18 , wherein the machine readable instructions, when executed by the at least one processor, cause the at least one computing device to at least:
 input at least one baseline parameter for the respective one of the plurality of available GPUs to determine the respective predicted interference. 
 
     
     
       20. The non-transitory computer-readable medium of  claim 15 , wherein the interference function identifies a one-to-one interference between two workloads, and the predicted interference identified as an output from the interference function corresponds to a worst-case predicted interference among at least one interference calculated for the at least one workload currently assigned to the particular GPU.