Digital data processing system for efficiently storing, moving, and/or processing data across a plurality of computing clusters

Abstract

A method, apparatus, computer-readable medium, and/or system described herein may be used to efficiently store, move, and/or process data across a plurality of computing clusters. For example, a computing device may receive an indication of one or more data storage locations within a first cluster of servers and/or an indication of one or more data storage locations within a second cluster of servers. The computing device may generate a data file comprising the indication of the one or more data storage locations within the first cluster of servers and/or the indication of one or more data storage locations within the second cluster of servers. Based on the generated data file, the computing device may generate a job to move data stored at the one or more data storage locations within the first cluster of servers to the one or more data storage locations within the second cluster of servers. Based on the job, the computing device may transmit, e.g., to the first cluster of servers and/or the second cluster of servers, instructions to move data stored at the one or more data storage locations within the first cluster of servers to the one or more data storage locations within the second cluster of servers.

What is claimed is: 
     
       1. An apparatus comprising:
 one or more processors; and 
 memory storing computer-executable instructions that, when executed by the one or more processors, cause the apparatus to:
 receive, from an authenticated user of a user device, an indication of a data storage location within a first cluster of servers; 
 send, to the user device and based on the data storage location, an indication of a second plurality of clusters of servers to which the data can be moved, wherein the indication comprises a destination file within an available environment of the second plurality of servers; 
 generate, based on a determination that the user is authorized to access one of the first cluster of servers or the second cluster of servers, a data file corresponding to the indication of the data storage location within the first cluster of servers and the destination file; 
 generate, based on the data file, a job to move data stored at the data storage location within the first cluster of servers to a data storage location corresponding to the destination file within the second cluster of servers; and 
 send, based on the job, to the first cluster of servers and the second cluster of servers, instructions to move data stored at the data storage location within the first cluster of servers to the one or more data storage locations within the second cluster of servers. 
 
 
     
     
       2. The apparatus of  claim 1 , wherein the instructions, when executed, cause the apparatus to:
 determine a format of the data stored at the data storage location, and wherein the data file comprises the indication of the data storage location within the first cluster of servers, the indication of the data storage location within the second cluster of servers, and the format of the data. 
 
     
     
       3. The apparatus of  claim 1 , wherein the data storage location within the first cluster of servers comprises one or more files within the first cluster of servers, and wherein the data storage location within the second cluster of servers comprises one or more files within the second cluster of servers. 
     
     
       4. The apparatus of  claim 1 , wherein the instructions, when executed, cause the apparatus to:
 receive from the user device, a plurality of source paths associated with the data storage location within the first cluster of servers; and 
 receive, from the user device, a plurality of destination paths associated with the data storage location within the second cluster of servers. 
 
     
     
       5. The apparatus of  claim 1 , wherein the instructions, when executed, cause the apparatus to:
 generate an identifier for the job to move data stored at the data storage location within the first cluster of servers to the data storage location within the second cluster of servers; and 
 send, to the user device, the identifier. 
 
     
     
       6. The apparatus of  claim 1 , wherein the job is generated based on a data type of the data stored at the data storage location within the first cluster of servers. 
     
     
       7. A method comprising:
 receiving, by a computing device from an authenticated user of a user device, an indication of a data storage location within a first cluster of servers; 
 sending, to the user device and based on the data storage location, an indication of a second plurality of clusters of servers to which the data can be moved, wherein the indication comprises a destination file within an available environment of the second plurality of servers; 
 generating, based on a determination that the user is authorized to access one of the first cluster of servers or the second cluster of servers, a data file corresponding to the indication of the data storage location within the first cluster of servers and the destination file; 
 generating, based on the data file, a job to move data stored at the data storage location within the first cluster of servers to a data storage location corresponding to the destination file within the second cluster of servers; and 
 sending, by the computing device based on the job, to the first cluster of servers and the second cluster of servers, instructions to move data stored at the data storage location within the first cluster of servers to the one or more data storage locations within the second cluster of servers. 
 
     
     
       8. The method of  claim 7  comprising:
 determining a format of the data stored at the data storage location, and wherein the data file comprises the indication of the data storage location within the first cluster of servers, the indication of the data storage location within the second cluster of servers, and the format of the data. 
 
     
     
       9. The method of  claim 7 , wherein the data storage location within the first cluster of servers comprises one or more files within the first cluster of servers, and wherein the data storage location within the second cluster of servers comprises one or more files within the second cluster of servers. 
     
     
       10. The method of  claim 7  comprising:
 receiving from the user device, a plurality of source paths associated with the data storage location within the first cluster of servers; and 
 receiving, from the user device, a plurality of destination paths associated with the data storage location within the second cluster of servers. 
 
     
     
       11. The method of  claim 7  comprising:
 generating an identifier for the job to move data stored at the data storage location within the first cluster of servers to the data storage location within the second cluster of servers; and 
 sending, to the user device, the identifier. 
 
     
     
       12. The method of  claim 7  comprising:
 generating, by the computing device, the job based on a data type of the data stored at the data storage location within the first cluster of servers. 
 
     
     
       13. A non-transitory computer readable medium storing instructions that, executed read by a computing device, causes the computing device to:
 receive, from an authenticated user of a user device, an indication of a data storage location within a first cluster of servers; 
 send, to the user device and based on the data storage location, an indication of a second plurality of clusters of servers to which the data can be moved, wherein the indication comprises a destination file within an available environment of the second plurality of servers; 
 generate, based on a determination that the user is authorized to access one of the first cluster of servers or the second cluster of servers, a data file corresponding to the indication of the data storage location within the first cluster of servers and the destination file; 
 generate, based on the data file, a job to move data stored at the data storage location within the first cluster of servers to a data storage location corresponding to the destination file within the second cluster of servers; and 
 send, based on the job, to the first cluster of servers and the second cluster of servers, instructions to move data stored at the data storage location within the first cluster of servers to the one or more data storage locations within the second cluster of servers. 
 
     
     
       14. The non-transitory computer readable medium of  claim 13 , wherein the instructions, when executed, cause the computing device to:
 determine a format of the data stored at the data storage location, and wherein the data file comprises the indication of the data storage location within the first cluster of servers, the indication of the data storage location within the second cluster of servers, and the format of the data. 
 
     
     
       15. The non-transitory computer readable medium of  claim 13 , wherein the data storage location within the first cluster of servers comprises one or more files within the first cluster of servers, and wherein the data storage location within the second cluster of servers comprises one or more files within the second cluster of servers. 
     
     
       16. The non-transitory computer readable medium of  claim 13 , wherein the instructions, when executed, cause the computing device to:
 receive from the user device, a plurality of source paths associated with the data storage location within the first cluster of servers; and 
 receive, from the user device, a plurality of destination paths associated with the data storage location within the second cluster of servers. 
 
     
     
       17. The non-transitory computer readable medium of  claim 13 , wherein the instructions, when executed, cause the computing device to:
 generate an identifier for the job to move data stored at the data storage location within the first cluster of servers to the data storage location within the second cluster of servers; and 
 send, to the user device, the identifier. 
 
     
     
       18. The non-transitory computer readable medium of  claim 13 , wherein the job is generated based on a data type of the data stored at the data storage location within the first cluster of servers.