Optimization of path selection for transfers of files

Abstract

A scheduling system for scheduling executions of tasks within a distributed computing system may include a file transfer manager configured to determine a file for transfer from a source location to a target location, the file being associated with file metadata characterizing the file, and with an organization. The file transfer manager may include an orchestrator configured to determine at least two transfer paths for the transfer, including at least a first transfer path utilizing a private wide area network (WAN) of the organization and a second transfer path utilizing a publicly available data hosting service, access transfer metadata characterizing the at least two transfer paths, and access organizational metadata characterizing organizational transfer path usage factors. The file transfer manager also may include a heuristics engine configured to execute path decision logic using the file metadata, the transfer metadata, and the organizational metadata, to thereby select a selected transfer path from the at least two transfer paths.

What is claimed is: 
     
         1 . A non-transitory computer-readable medium storing executable instructions that when executed by at least one processor are configured to cause the at least one processor to:
 intercept an operating system call issued at a source computing device, the intercepted operating system call including a file transfer request for transferring a file;   programmatically select one of a private wide area network (WAN) or a data hosting service for transferring the file based on metadata associated with the file, the private WAN including a computer network administered by an organization, the data hosting service including a computer network that is not administered by the organization;   transfer the file from the source computing device to a target computing device using the private WAN when the private WAN is selected; and   transfer the file from the source computing device to the target computing device using the data hosting service when the data hosting service is selected, including:
 upload the file to the data hosting service; and 
 instruct a data center to download the file from the data hosting service. 
   
     
     
         2 . The non-transitory computer-readable medium of  claim 1 , wherein the metadata includes a size of the file, wherein the data hosting service is selected when the size of the file is greater than a threshold amount. 
     
     
         3 . The non-transitory computer-readable medium of  claim 1 , further comprising:
 determine whether or not the file already exists at the target computing device based on one or more checksum values associated with the file.   
     
     
         4 . The non-transitory computer-readable medium of  claim 1 , wherein the metadata includes a security level, a priority level related to individual members or classes of members of the organization associated with the file, or at least one constraint on file transfers. 
     
     
         5 . The non-transitory computer-readable medium of  claim 1 , wherein the file is intercepted via an application programming interface (API). 
     
     
         6 . The non-transitory computer-readable medium of  claim 1 , further comprising:
 compute network metadata about network conditions of at least one of the private WAN or the data hosting service,   wherein the private WAN or the data hosting service is selected based on the metadata and the network metadata.   
     
     
         7 . The non-transitory computer-readable medium of  claim 1 , wherein the private WAN is associated with geographically-distributed datacenters including a local file store proximate to the source computing device and a remote file store proximate to the target computing device. 
     
     
         8 . The non-transitory computer-readable medium of  claim 1 , further comprising:
 transfer configuration data to the data center, the configuration data including information for accessing the data hosting service.   
     
     
         9 . The non-transitory computer-readable medium of  claim 8 , wherein the configuration data includes credential information associated with the organization for use in accessing an account provided by the data hosting service 
     
     
         10 . The non-transitory computer-readable medium of  claim 1 , wherein the private WAN or the data hosting service is selected based on path decision logic, the path decision logic specifying one or more weights applied to the metadata. 
     
     
         11 . The non-transitory computer-readable medium of  claim 10 , wherein the metadata includes security-related metadata and transfer latency metadata, wherein the security-related metadata is associated with a higher weight than the transfer latency metadata. 
     
     
         12 . The non-transitory computer-readable medium of  claim 1 , further comprising:
 programmatically select one of a plurality of data hosting services for transferring the file.   
     
     
         13 . A method comprising:
 intercepting an operating system call issued at a source computing device, the intercepted operating system call including a file transfer request for transferring a file;   programmatically selecting one of a private wide area network (WAN) or a data hosting service for transferring the file based on metadata associated with the file, the private WAN including a computer network administered by an organization, the data hosting service including a computer network that is not administered by the organization;   transferring the file from the source computing device to a target computing device using the private WAN when the private WAN is selected; and   transferring the file from the source computing device to the target computing device using the data hosting service when the data hosting service is selected, including:
 uploading the file to the data hosting service; and 
 instructing a data center to download the file from the data hosting service. 
   
     
     
         14 . The method of  claim 13 , wherein the metadata includes a size of the file, wherein the data hosting service is selected when the size of the file is greater than a threshold amount. 
     
     
         15 . The method of  claim 13 , further comprising:
 determining whether or not the file already exists at the target computing device based on one or more checksum values associated with the file.   
     
     
         16 . The method of  claim 13 , wherein the metadata includes a security level, a priority level related to individual members or classes of members of the organization associated with the file, or at least one constraint on file transfers. 
     
     
         17 . The method of  claim 13 , wherein the file is intercepted via an application programming interface (API). 
     
     
         18 . The method of  claim 13 , further comprising:
 computing network metadata about network conditions of at least one of the private WAN or the data hosting service,   wherein the private WAN or the data hosting service is selected based on the metadata and the network metadata.   
     
     
         19 . The method of  claim 13 , wherein the private WAN is associated with geographically-distributed datacenters including a local file store proximate to the source computing device and a remote file store proximate to the target computing device. 
     
     
         20 . A system for file transfer, the system comprising:
 at least one processor;   a non-transitory computer-readable medium storing executable instructions that when executed by the at least one processor are configured to cause the at least one processor to execute a file manager, the file manager configured to:   intercept an operating system call issued at a source computing device, the intercepted operating system call including a file transfer request for transferring a file;   programmatically select one of a private wide area network (WAN) or a data hosting service for transferring the file based on metadata associated with the file, the private WAN including a computer network administered by an organization, the data hosting service including a computer network that is not administered by the organization;   transfer the file from the source computing device to a target computing device using the private WAN when the private WAN is selected; and   transfer the file from the source computing device to the target computing device using the data hosting service when the data hosting service is selected, including:
 upload the file to the data hosting service; and 
   instruct a data center to download the file from the data hosting service.