Distribution of job in a portable format in distributed computing environments

Abstract

A distributed computing system includes a device that creates a job to be performed by one or more workers, where the job includes a group of tasks. The device further encodes the job into a group of portable format files, where each portable format file includes one or more tasks of the group of tasks. Each portable format file may be handled by each scheduler of a group of different schedulers. The device also transfers the group of portable format files via a network of the distributed computing system and sends information related to the job to one scheduler of the group of different schedulers. The information allows the one scheduler to cause the group of portable format files to be distributed to the one or more workers.

What is claimed is: 
     
       1. A non-transitory computer-readable medium that stores instructions, the instructions comprising:
 one or more instructions which, when executed by a processor, cause the processor to:
 identify an available scheduler in a distributed computing environment that includes:
 a client, 
 a plurality of schedulers,
 at least two of the plurality of schedulers being of different types, and 
 
 a plurality of workers; 
 
 identify an adapter compatible with the identified scheduler and the client,
 the adapter providing information relating to one or more properties or functions of the identified scheduler; 
 
 create a job object defining a job to be executed by at least one worker in the distributed computing environment; 
 encode the job object into a portable format that is independent of an architecture of the plurality of schedulers; 
 transfer the job object encoded in the portable format via a network of the distributed computing environment using the information from the adapter; and 
 send information allowing the identified scheduler to distribute the job object in the portable format to at least one worker of the plurality of workers,
 the information being sent to the identified scheduler. 
 
 
 
     
     
       2. The computer-readable medium of  claim 1 , where the job object includes at least one task. 
     
     
       3. The computer-readable medium of  claim 1 , where the job object includes license information. 
     
     
       4. The computer-readable medium of  claim 1 , where the portable format includes a binary format. 
     
     
       5. The computer-readable medium of  claim 1 , where the information includes one or more properties of the job. 
     
     
       6. The computer-readable medium of  claim 1 , where the instructions further include:
 one or more instructions to break the job down into a plurality of portable format files,
 each portable format file, of the plurality of portable format files, including one or more tasks, and 
 
 the one or more instructions to transfer the job object encoded in the portable format include:
 one or more instructions to transfer the plurality of portable format files. 
 
 
     
     
       7. A method comprising:
 identifying an available scheduler, of a plurality of schedulers in a distributed computing environment,
 at least two of the plurality of schedulers being of different types, and 
 the identifying being performed by a computing device; 
 
 identifying an adapter compatible with the identified scheduler and a client,
 the adapter providing information relating to one or more properties or functions of the identified scheduler, and 
 the identifying the adapter being performed by the computing device; 
 
 creating a job to be performed by one or more workers of a plurality of workers,
 the job including a plurality of tasks, and 
 the creating being performed by the computing device; 
 
 forming a plurality of portable format files associated with the job,
 each portable format file, of the plurality of portable format files, including one or more tasks of the plurality of tasks, 
 each portable format file, of the plurality of portable format files, being capable of being handled by any scheduler of the plurality of schedulers, and 
 the forming being performed by the computing device; 
 
 transferring the plurality of portable format files via a network of the distributed computing environment using the information from the adapter,
 the transferring being performed by the computing device; and 
 
 sending information allowing the identified scheduler to distribute the plurality of portable format files to the one or more workers,
 the information allowing the identified scheduler to distribute the portable format files being sent to the identified scheduler, and 
 the sending being performed by the computing device. 
 
 
     
     
       8. The method of  claim 7 , where the plurality of different schedulers include schedulers provided by different vendors. 
     
     
       9. The method of  claim 7 , where at least one of the portable format files includes license information. 
     
     
       10. The method of  claim 7 , where each portable format file includes binary data. 
     
     
       11. The method of  claim 7 , where the information includes one or more properties of the job. 
     
     
       12. The method of  claim 7 , further comprising:
 receiving a portable format file, of the plurality of portable format files, at a worker of the one or more workers; 
 restoring the one or more tasks from the portable format file; and 
 executing the one or more restored tasks. 
 
     
     
       13. A distributed computing system including:
 a device, implemented at least partially in hardware, to:
 identify an available scheduler in a distributed computing environment that includes:
 a client, 
 a plurality of schedulers,
 at least two of the plurality of schedulers being of different types, and 
 
 a plurality of workers; 
 
 identify an adapter compatible with the identified scheduler and the client,
 the adapter providing information relating to one or more properties or functions of the identified scheduler; 
 
 create a job to be performed by one or more of the workers,
 the job including a plurality of tasks; 
 
 encode the job into a plurality of portable format files,
 each portable format file, of the plurality of portable format files, including one or more tasks of the plurality of tasks, and 
 each portable format file being capable of being handled by each scheduler of the plurality of schedulers; 
 
 transfer the plurality of portable format files to a network of the distributed computing system using the information from the adapter; and 
 send information allowing the identified scheduler to distribute the plurality of portable format files to the one or more workers,
 the information being sent to the identified scheduler. 
 
 
 
     
     
       14. The distributed computing system of  claim 13 , where the plurality of different schedulers includes schedulers provided by different vendors. 
     
     
       15. The distributed computing system of  claim 13 , where each portable format file, of the plurality of portable format files, includes license information. 
     
     
       16. The distributed computing system of  claim 13 , where the information includes one or more properties of the job. 
     
     
       17. The distributed computing system of  claim 13 , where a worker, of the one or more workers, is to:
 receive a portable format file, 
 decode the portable format file to restore the one or more tasks, and 
 execute the one or more restored tasks. 
 
     
     
       18. The distributed computing system of  claim 13 , where the plurality of portable format files include binary data. 
     
     
       19. A method comprising:
 identifying an available scheduler in a distributed computing environment that includes a client, a plurality of schedulers, and a plurality of workers,
 the identifying being performed by a computing device; 
 
 identifying an adapter compatible with the identified scheduler and the client,
 the adapter providing information relating to one or more properties or functions of the identified scheduler, and 
 the identifying the adapter being performed by the computing device; 
 
 creating a job that includes at least one task,
 the creating being performed by the computing device; 
 
 encoding the job into a portable format,
 the portable format being independent of a platform structure of the identified scheduler, and 
 the encoding being performed by the computing device; 
 
 transferring the encoded job in the portable format via a network using the information from the adapter,
 the transferring being performed by the computing device; and 
 
 sending information allowing the identified scheduler to distribute the encoded job in the portable format to a worker,
 the information being sent to the identified scheduler, 
 the sending being performed by the computing device, 
 the encoded job in the portable format being received at the worker, 
 the encoded job being decoded, from the portable format, to restore the job, and 
 the restored job being executed at the worker. 
 
 
     
     
       20. The method of  claim 19 , where the portable format includes a binary format. 
     
     
       21. The method of  claim 19 , where the portable format includes a serial format. 
     
     
       22. A device comprising:
 a memory to store instructions; and 
 a processor to execute the instructions to:
 identify an available scheduler, of a plurality of different schedulers in a distributed computing environment; 
 identify an adapter compatible with the identified scheduler and the client,
 the adapter providing information relating to one or more properties or functions of the identified scheduler; 
 
 create a task to be performed by a worker of the plurality of workers; 
 form a portable format file from the task,
 the portable format file including the task and being independent of an architecture of the plurality of different schedulers; 
 
 transfer the portable format file via a network of the distributed computing environment using the information from the adapter; and 
 send information allowing the identified scheduler, of the plurality of different schedulers, to distribute the portable format file to at least one worker of the plurality of workers,
 the information allowing the identified scheduler to distribute the portable format file. 
 
 
 
     
     
       23. A non-transitory computer-readable medium that stores instructions, the instructions comprising:
 one or more instructions which, when executed by a processor, cause the processor to:
 identify a scheduler in a distributed computing environment; 
 identify an adapter compatible with the identified scheduler and a client,
 the adapter providing information relating to one or more properties or functions of the identified scheduler; 
 
 create a job object defining a job to be executed by at least one worker; 
 encode the job object into a portable format that is independent of an architecture of the identified scheduler; and 
 transfer the job object, encoded in the portable format, via a network of the distributed computing environment using the information from the adapter. 
 
 
     
     
       24. The computer-readable medium of  claim 23 , where the job object includes at least one task. 
     
     
       25. The computer-readable medium of  claim 23 , where the job object includes license information. 
     
     
       26. The computer-readable medium of  claim 23 , where the portable format includes a binary format. 
     
     
       27. The computer-readable medium of  claim 23 , where the information includes one or more properties of the job. 
     
     
       28. The computer-readable medium of  claim 23 , where the instructions further include:
 one or more instructions to break the job down into a plurality of portable format files,
 each portable format file, of the plurality of portable format files, including one or more tasks, and 
 
 where the one or more instructions to transfer the job object encoded in the portable format include:
 one or more instructions to transfer the plurality of portable format files.