Adaptive data striping and replication across multiple storage clouds for high availability and performance

Abstract

Technologies for adaptively striping data across multiple storage clouds include receiving user constraints corresponding one or more cloud storage providers, receiving a file to be striped across the cloud storage providers, splitting the received file into file blocks, allocating each of the file blocks to a different one of the cloud storage providers as a function of the user constraints and operating conditions of each of the cloud storage providers, and sending each of the file blocks to the cloud storage provider to which each file block is allocated. In some embodiments, file blocks may be re-allocated from one cloud storage provider to another cloud storage provider as a function of changing user constraints or operating conditions. In addition, each of the file blocks may be retrieved from the cloud storage providers to re-assemble the file.

1 . A computing device for re-assembling a file from a plurality of file blocks striped across a plurality of storage clouds, the computing device comprising:
 a block location determination module to (i) determine that a first cloud storage provider of a plurality of cloud storage providers is storing a first file block of a particular file to be re-assembled, and (ii) determine that a second cloud storage provider of a plurality of cloud storage providers is storing a second file block of the particular file to be re-assembled;   a file re-assembly module to (i) request the first file block from the first cloud storage provider, and (ii) request the second file block from the second cloud storage provider; and   a communication module to (i) receive the first file block from the first cloud storage provider, and (ii) receive the second file block from the second cloud storage provider,   wherein the file re-assembly module is further to re-assemble the particular file to be re-assembled from the received first and second file blocks.   
     
     
         2 . The computing device of  claim 1 , wherein the file re-assembly module further to determine whether the second file block of the particular file to be re-assembled exists. 
     
     
         3 . The computing device of  claim 1 , wherein the block location determination module further to:
 determine whether location metadata corresponding to the first file block is stored in a local metadata storage of the computing device; and   determine whether location metadata corresponding to the second file block is stored in the local metadata storage of the computing device.   
     
     
         4 . The computing device of  claim 3 , wherein the block location determination module further to:
 request the location metadata corresponding to the first file block from a cloud resource allocation server in response to determining that the location metadata corresponding to the first file block is not stored in the local metadata storage of the computing device; and   request the location metadata corresponding to the second file block from the cloud resource allocation server in response to determining that the location metadata corresponding to the second file block is not stored in the local metadata storage of the computing device.   
     
     
         5 . The computing device of  claim 4 , wherein the block location determination module further to:
 request the location metadata corresponding to the first file block from a second cloud resource allocation server to which the location metadata has been replicated; and   request the location metadata corresponding to the second file block from the second cloud resource allocation server to which the location metadata has been replicated.   
     
     
         6 . The computing device of  claim 4 , wherein to determine that a first cloud storage provider of a plurality of cloud storage providers is storing a first file block of a particular file to be re-assembled comprises to determine that a first cloud storage provider of a plurality of cloud storage providers is storing a first file block of a particular file to be re-assembled as a function of the location metadata corresponding to the first file block, and
 wherein to determine that a second cloud storage provider of a plurality of cloud storage providers is storing the second file block comprises to determine that a second cloud storage provider of a plurality of cloud storage providers is storing the second file block as a function of the location metadata corresponding to the second file block.   
     
     
         7 . The computing device of  claim 3 , wherein the block location determination module further to:
 determine, in response to determining that the location metadata corresponding to the first file block is stored in the local metadata storage, whether the location metadata corresponding to the first file block stored in the local metadata storage is outdated; and   request, in response to determining that the location metadata corresponding to the first file block stored in the local metadata storage is outdated, the location metadata corresponding to the first file block from a cloud resource allocation server.   
     
     
         8 . The computing device of  claim 1 , wherein the file reassembly module further to determine whether re-assembly metadata corresponding to the particular file to be re-assembled is stored in the local metadata storage of the computing device. 
     
     
         9 . The computing device of  claim 8 , wherein the file reassembly module further to request the re-assembly metadata corresponding to the particular file to be re-assembled from a cloud resource allocation server in response to determining that the re-assembly metadata corresponding to the particular file to be re-assembled is not stored in the local metadata storage of the computing device,
 wherein to re-assemble the particular file to be re-assembled from the received first and second file blocks comprises to re-assemble the particular file to be re-assembled from the received first and second file blocks as a function of the re-assembly metadata.   
     
     
         10 . A method for re-assembling a file from a plurality of file blocks striped across a plurality of storage clouds by a computing device, the method comprising:
 determining, by the computing device, that a first cloud storage provider of a plurality of cloud storage providers is storing a first file block of a particular file to be re-assembled;   determining, by the computing device, that a second cloud storage provider of a plurality of cloud storage providers is storing a second file block of the particular file to be re-assembled;   requesting, by the computing device, the first file block from the first cloud storage provider;   requesting, by the computing device, the second file block from the second cloud storage provider;   receiving, by the computing device, the first file block from the first cloud storage provider;   receiving, by the computing device, the second file block from the second cloud storage provider; and   re-assembling, by the computing device, the particular file to be re-assembled from the received first and second file blocks.   
     
     
         11 . The method of  claim 10 , further comprising:
 determining, by the computing device, whether location metadata corresponding to the first file block is stored in a local metadata storage of the computing device; and   determining, by the computing device, whether location metadata corresponding to the second file block is stored in the local metadata storage of the computing device.   
     
     
         12 . The method of  claim 11 , further comprising:
 requesting, by the computing device, the location metadata corresponding to the first file block from a cloud resource allocation server in response to determining that the location metadata corresponding to the first file block is not stored in the local metadata storage of the computing device; and   requesting, by the computing device, the location metadata corresponding to the second file block from the cloud resource allocation server in response to determining that the location metadata corresponding to the second file block is not stored in the local metadata storage of the computing device.   
     
     
         13 . The method of  claim 12 , further comprising:
 requesting, by the computing device, the location metadata corresponding to the first file block from a second cloud resource allocation server to which the location metadata has been replicated; and   requesting, by the computing device, the location metadata corresponding to the second file block from the second cloud resource allocation server to which the location metadata has been replicated.   
     
     
         14 . The method of  claim 12 , wherein determining that a first cloud storage provider of a plurality of cloud storage providers is storing a first file block of a particular file to be re-assembled comprises determining that a first cloud storage provider of a plurality of cloud storage providers is storing a first file block of a particular file to be re-assembled as a function of the location metadata corresponding to the first file block, and
 wherein determining that a second cloud storage provider of a plurality of cloud storage providers is storing the second file block comprises determining that a second cloud storage provider of a plurality of cloud storage providers is storing the second file block as a function of the location metadata corresponding to the second file block.   
     
     
         15 . The method of  claim 11 , further comprising:
 determining, by the computing device and in response to determining that the location metadata corresponding to the first file block is stored in the local metadata storage, whether the location metadata corresponding to the first file block stored in the local metadata storage is outdated; and   requesting, by the computing device and in response to determining that the location metadata corresponding to the first file block stored in the local metadata storage is outdated, the location metadata corresponding to the first file block from a cloud resource allocation server.   
     
     
         16 . The method of  claim 10 , further comprising determining, by the computing device, whether re-assembly metadata corresponding to the particular file to be re-assembled is stored in the local metadata storage of the computing device. 
     
     
         17 . The method of  claim 16 , further comprising requesting, by the computing device, the re-assembly metadata corresponding to the particular file to be re-assembled from a cloud resource allocation server in response to determining that the re-assembly metadata corresponding to the particular file to be re-assembled is not stored in the local metadata storage of the computing device,
 wherein re-assembling the particular file to be re-assembled from the received first and second file blocks comprises re-assembling the particular file to be re-assembled from the received first and second file blocks as a function of the re-assembly metadata.   
     
     
         18 . One or more machine readable media comprising a plurality of instructions stored thereon that in response to being executed result in a computing device:
 determining, by the computing device, that a first cloud storage provider of a plurality of cloud storage providers is storing a first file block of a particular file to be re-assembled;   determining, by the computing device, that a second cloud storage provider of a plurality of cloud storage providers is storing a second file block of the particular file to be re-assembled;   requesting, the first file block from the first cloud storage provider;   requesting the second file block from the second cloud storage provider;   receiving the first file block from the first cloud storage provider;   receiving the second file block from the second cloud storage provider; and   re-assembling the particular file to be re-assembled from the received first and second file blocks.   
     
     
         19 . The one or more computer readable media of  claim 18 , wherein the plurality of instructions further result in the computing device:
 determining whether location metadata corresponding to the first file block is stored in a local metadata storage of the computing device; and   determining whether location metadata corresponding to the second file block is stored in the local metadata storage of the computing device.   
     
     
         20 . The one or more computer readable media of  claim 19 , wherein the plurality of instructions further result in the computing device:
 requesting the location metadata corresponding to the first file block from a cloud resource allocation server in response to determining that the location metadata corresponding to the first file block is not stored in the local metadata storage of the computing device; and   requesting the location metadata corresponding to the second file block from the cloud resource allocation server in response to determining that the location metadata corresponding to the second file block is not stored in the local metadata storage of the computing device.   
     
     
         21 . The one or more computer readable media of  claim 20 , wherein the plurality of instructions further result in the computing device:
 requesting the location metadata corresponding to the first file block from a second cloud resource allocation server to which the location metadata has been replicated; and   requesting the location metadata corresponding to the second file block from the second cloud resource allocation server to which the location metadata has been replicated.   
     
     
         22 . The one or more computer readable media of  claim 20 , wherein determining that a first cloud storage provider of a plurality of cloud storage providers is storing a first file block of a particular file to be re-assembled comprises determining that a first cloud storage provider of a plurality of cloud storage providers is storing a first file block of a particular file to be re-assembled as a function of the location metadata corresponding to the first file block, and
 wherein determining that a second cloud storage provider of a plurality of cloud storage providers is storing the second file block comprises determining that a second cloud storage provider of a plurality of cloud storage providers is storing the second file block as a function of the location metadata corresponding to the second file block.   
     
     
         23 . The one or more computer readable media of  claim 19 , wherein the plurality of instructions further result in the computing device:
 determining, in response to determining that the location metadata corresponding to the first file block is stored in the local metadata storage, whether the location metadata corresponding to the first file block stored in the local metadata storage is outdated; and   requesting, in response to determining that the location metadata corresponding to the first file block stored in the local metadata storage is outdated, the location metadata corresponding to the first file block from a cloud resource allocation server.   
     
     
         24 . The one or more computer readable media of  claim 18 , wherein the plurality of instructions further result in the computing device determining whether re-assembly metadata corresponding to the particular file to be re-assembled is stored in the local metadata storage of the computing device. 
     
     
         25 . The one or more computer readable media of  claim 24 , wherein the plurality of instructions further result in the computing device requesting the re-assembly metadata corresponding to the particular file to be re-assembled from a cloud resource allocation server in response to determining that the re-assembly metadata corresponding to the particular file to be re-assembled is not stored in the local metadata storage of the computing device,
 wherein re-assembling the particular file to be re-assembled from the received first and second file blocks comprises re-assembling the particular file to be re-assembled from the received first and second file blocks as a function of the re-assembly metadata.