Data transfer between storage systems using data fingerprints

Abstract

A system and method for data replication is described. A destination storage system receives a message from a source storage system as part of a replication process. The message includes an identity of a first file, information about where the first file is stored in the source storage system, a name of a first data being used by the first file and stored at a first location of the source storage system, and a fingerprint of the first data. The destination storage system determines that a mapping database is unavailable or inaccurate, and accesses a fingerprint database using the fingerprint of the first data received with the message to determine whether data stored in the destination storage system has a fingerprint identical to the fingerprint of the first data.

What is being claimed is: 
     
         1 . A method comprising:
 receiving a message comprising an identity of a first file, a file storage location of where the first file is stored in a first storage system, a name of a first data used by the first file, and a fingerprint of the first data, wherein the first data is stored at a first location of the first storage system;   determining that a mapping database of a second storage system, which maps storage locations of data in the first storage system to storage locations of replicated data in the second storage system, is unavailable; and   accessing a fingerprint database of the second storage system using the fingerprint to determine whether replicated data stored in the second storage system has a second fingerprint identical to the fingerprint of the first data, wherein the fingerprint database stores entries of fingerprints of replicated data stored by the second storage system, wherein an entry of a first replicated data comprises a corresponding fingerprint for the first replicated data, a corresponding identity of a file at the first storage system that uses the first replicated data, and a corresponding file storage location of where the file is stored in the first storage system.   
     
     
         2 . The method of  claim 1 , wherein the corresponding file storage location, within the entry of the fingerprint database, corresponds to an offset value and a length value. 
     
     
         3 . The method of  claim 1 , wherein the fingerprint of the first data corresponds to a checksum value generated by the first storage system using a checksum function. 
     
     
         4 . The method of  claim 1 , comprising:
 transmitting, to the first storage system, a request message to confirm whether the first data is identical to the replicated data having the second fingerprint.   
     
     
         5 . The method of  claim 1 , wherein accessing the fingerprint database includes performing a lookup in the fingerprint database by comparing the fingerprint of the first data with the entries of fingerprints stored in the fingerprint database. 
     
     
         6 . The method of  claim 4 , wherein the request message includes a second identity of a second file that uses the replicated data. 
     
     
         7 . The method of  claim 4 , comprising:
 receiving a response message indicating that the first data is identical to the replicated data; and   updating the mapping database by associating the name of the first data stored at the first location by the first storage system with a local location where the replicated data is stored by the second storage system.   
     
     
         8 . The method of  claim 7 , comprising:
 updating the fingerprint database to include a new entry for the first file, the new entry comprising the fingerprint of the first data, the identity of the first file, and the file storage location of where the first file is stored.   
     
     
         9 . The method of  claim 4 , further comprising:
 receiving a response message indicating that the first data is not identical to the replicated data;   receiving the first file and the first data;   storing the first data within a local location in the second storage system;   updating the mapping database by associating the name of the first data stored at the first location of the first storage system with the local location where the first data is stored in the second storage system; and   updating the fingerprint database to include a new entry for the first file, the new entry including the fingerprint of the first data, the identity of the first file, and the file storage location of where the first file is stored in the first storage system.   
     
     
         10 . A non-transitory computer-readable medium comprising instructions that, when executed by a processor, cause the processor to perform operations comprising:
 receiving a message comprising an identity of a first file, a file storage location of where the first file is stored in a first storage system, a name of a first data used by the first file, and a fingerprint of the first data, wherein the first data is stored at a first location of the first storage system;   determining that a mapping database of a second storage system, which maps storage locations of data in the first storage system to storage locations of replicated data in the second storage system, is unavailable; and   accessing a fingerprint database of the second storage system using the fingerprint to determine whether replicated data stored in the second storage system has a second fingerprint identical to the fingerprint of the first data, wherein the fingerprint database stores entries of fingerprints of replicated data stored by the second storage system, wherein an entry of a first replicated data comprises a corresponding fingerprint for the first replicated data, a corresponding identity of a file at the first storage system that uses the first replicated data, and a corresponding file storage location of where the file is stored in the first storage system.   
     
     
         11 . The non-transitory computer-readable medium of  claim 10 , wherein the file storage location corresponds to an offset value and a length value associated with the first storage system. 
     
     
         12 . The non-transitory computer-readable medium of  claim 10 , wherein the fingerprint of the first data corresponds to a checksum value generated by the first storage system using a checksum function. 
     
     
         13 . The non-transitory computer-readable medium of  claim 10 , wherein the corresponding file storage location, within the entry of the fingerprint database, comprises an offset value and a length value associated with the first storage system. 
     
     
         14 . The non-transitory computer-readable medium of  claim 10 , wherein the operations comprise:
 accessing the fingerprint database by performing a lookup in the fingerprint database by comparing the fingerprint of the first data with the entries of fingerprints stored in the fingerprint database.   
     
     
         15 . The non-transitory computer-readable medium of  claim 10 , wherein the operations comprise:
 transmitting, to the first storage system, a request message to confirm whether the first data is identical to the replicated data having the second fingerprint.   
     
     
         16 . The non-transitory computer-readable medium of  claim 10 , wherein the mapping database comprises a mapping entry mapping a source-side name, used by the first storage system to refer to the file, to a location of where replicated data of the file is stored by the second storage system. 
     
     
         17 . The non-transitory computer-readable medium of  claim 15 , wherein the operations comprise:
 receiving a response message indicating that the first data is identical to the replicated data;   updating the fingerprint database to include an entry for the first file, the entry comprising the fingerprint of the first data, the identity of the first file, and the file storage location of where the first file is stored based upon the response message.   
     
     
         18 . The non-transitory computer-readable medium of  claim 15 , wherein the operations comprise:
 receiving a response message indicating that the first data is not identical to the replicated data;   receiving the first file and the first data;   storing the first data within a local location in the second storage system;   updating the mapping database by associating the name of the first data stored at the first location of the first storage system with the local location where the first data is stored in the second storage system; and   updating the fingerprint database to include a new entry for the first file, the new entry including the fingerprint of the first data, the identity of the first file, and the file storage location of where the first file is stored in the first storage system.   
     
     
         19 . A computing device comprising:
 a memory resource storing instructions; and   at least one processor coupled to the memory resource, the at least one processor executing the instructions to perform operations comprising:
 receiving a message comprising an identity of a first file, a file storage location of where the first file is stored in a first storage system, a name of a first data used by the first file, and a fingerprint of the first data, wherein the first data is stored at a first location of the first storage system; 
 determining that a mapping database of a second storage system, which maps storage locations of data in the first storage system to storage locations of replicated data in the second storage system, is unavailable; and 
 accessing a fingerprint database of the second storage system using the fingerprint to determine whether replicated data stored in the second storage system has a second fingerprint identical to the fingerprint of the first data, wherein the fingerprint database stores entries of fingerprints of replicated data stored by the second storage system, wherein an entry of a first replicated data comprises a corresponding fingerprint for the first replicated data, a corresponding identity of a file at the first storage system that uses the first replicated data, and a corresponding file storage location of where the file is stored in the first storage system. 
   
     
     
         20 . The computing device of  claim 19 , wherein the corresponding file storage location, within the entry of the fingerprint database, corresponds to an offset value and a length value.