Data syncing in a distributed system

Abstract

Disclosed are systems, computer-readable mediums, and methods for receiving a start replication message to replicate a source volume to a replicated volume. A source system forwards I/O requests to the replica server. A data structure associated with the replicated volume is initialized. A write request is received from the source system. The write data is written to the replicated volume and the data structure is updated. Source metadata associated with the source volume is received. The source metadata is compared with prior metadata associated with a prior point-in-time image of the source volume to determine blocks of data that have changed since the prior point-in-time image of the source volume. A first block is determined to not be retrieved based upon the data structure. A second block is determined to be retrieved based upon the data structure. The second block is received and written to the replicated volume.

What is claimed is: 
     
         1 . A system comprising:
 a replica server comprising one or more processors configured to:
 receive a start replication message from a source system to replicate data of a source volume to a replicated volume on the replica server, wherein the replicated volume comprises a copy of data of the source volume, and wherein the source system forwards input/output (I/O) requests to the replica server after the start replication message is sent; 
 initiate a data structure associated with units of data of the replicated volume; 
 receive, from the source system, a write request comprising write data associated a unit of data of the replicated volume, wherein the source system wrote the write data to the source volume based upon the write request; 
 write the write data to the replicated volume; 
 update the data structure to indicate the write data has been written after the receipt of the start replication message; 
 receive source metadata associated with the source volume, wherein the metadata comprises an ordered list of block identifiers for data blocks of the source volume, and wherein each block identifier is used to access a data block; 
 compare the source metadata with prior metadata associated with a prior point-in-time image of the source volume to determine blocks of data that have changed since the prior point-in-time image of the source volume; 
 determine a first block of the blocks of data should not be retrieved based upon the data structure; 
 determine a second block of the blocks of data should be retrieved based upon the data structure; 
 retrieve the second block from the source system; and 
 write the second block to the replicated volume. 
   
     
     
         2 . The system of  claim 1 , wherein the one or more processors are further configured to send a replication complete message to the source system, wherein the source system no longer forwards I/O to the replica server based upon receipt of the replication complete message. 
     
     
         3 . The system of  claim 1 , wherein the one or more processors are further configured to:
 request verification data from the source system;   receive the verification data from the source system; and   compare the received verification data with corresponding verification data of the replica volume to determine the replication was successful.   
     
     
         4 . The system of  claim 1 , wherein a unit of data is a sub-block, wherein a block consists of a plurality of sub-blocks, and wherein to write the second block the one or more processors are further configured to:
 determine which sub-blocks have been updated since the receipt of the start replication message based upon the data structure; and   write sub-blocks of the second block based upon on the sub-block not being updated since the receipt of the start replication message.   
     
     
         5 . The system of  claim 1 , further comprising a data server, wherein the data server comprising one or more processors configured to:
 request quality of service parameters for the replicated volume;   request quality of service parameters for the source volume;   determine if the replication of the source volume to the replicated volume will succeed based upon the quality of service parameters of the replicated volume and the source volume.   
     
     
         6 . The system of  claim 5 , wherein the data server comprising one or more processors are further configured to determine a ratio of the quality of service parameters for the replicated volume to the quality of service parameters for the source volume, wherein the replication of the source volume will succeed if the ratio is greater than one. 
     
     
         7 . The system of  claim 1 , wherein the data blocks of the replicated volume are randomly and evenly distributed across a cluster containing the replicated volume. 
     
     
         8 . A method comprising:
 receiving, at a replica server, a start replication message from a source system to replicate data of a source volume to a replicated volume on the replica server, wherein the replicated volume comprises a copy of data of the source volume, and wherein the source system forwards input/output (I/O) requests to the replica server after the start replication message is sent;   initiating a data structure associated with units of data of the replicated volume;   receiving, from the source system, a write request comprising write data associated a unit of data of the replicated volume, wherein the source system wrote the write data to the source volume based upon the write request;   writing the write data to the replicated volume;   updating the data structure to indicate the write data has been written after the receipt of the start replication message;   receiving source metadata associated with the source volume, wherein the metadata comprises an ordered list of block identifiers for data blocks of the source volume, and wherein each block identifier is used to access a data block;   comparing the source metadata with prior metadata associated with a prior point-in-time image of the source volume to determine blocks of data that have changed since the prior point-in-time image of the source volume;   determining, using a processor, a first block of the blocks of data should not be retrieved based upon the data structure;   determining a second block of the blocks of data should be retrieved based upon the data structure;   retrieving the second block from the source system; and   writing the second block to the replicated volume.   
     
     
         9 . The method of  claim 8 , further comprising sending a replication complete message to the source system, wherein the source system no longer forwards I/O to the replica server based upon receipt of the replication complete message. 
     
     
         10 . The method of  claim 8 , further comprising:
 requesting verification data from the source system;   receiving the verification data from the source system; and   comparing the received verification data with corresponding verification data of the replica volume to determine the replication was successful.   
     
     
         11 . The method of  claim 8 , wherein a unit of data is a sub-block, wherein a block consists of a plurality of sub-blocks, and wherein the method further comprises:
 determining which sub-blocks have been updated since the receipt of the start replication message based upon the data structure; and   writing sub-blocks of the second block based upon on the sub-block not being updated since the receipt of the start replication message.   
     
     
         12 . The method of  claim 8 , further comprising
 requesting quality of service parameters for the replicated volume;   requesting quality of service parameters for the source volume;   determining if the replication of the source volume to the replicated volume will succeed based upon the quality of service parameters of the replicated volume and the source volume.   
     
     
         13 . The method of  claim 12 , further comprising determining a ratio of the quality of service parameters for the replicated volume to the quality of service parameters for the source volume, wherein the replication of the source volume will succeed if the ratio is greater than one. 
     
     
         14 . The method of  claim 8 , wherein the data blocks of the replicated volume are randomly and evenly distributed across a cluster containing the replicated volume. 
     
     
         15 . A non-transitory computer-readable medium having instructions stored thereon, the instructions comprising:
 instructions to receive a start replication message from a source system to replicate data of a source volume to a replicated volume on the replica server, wherein the replicated volume comprises a copy of data of the source volume, and wherein the source system forwards input/output (I/O) requests to the replica server after the start replication message is sent;   instructions to initiate a data structure associated with units of data of the replicated volume;   instructions to receive, from the source system, a write request comprising write data associated a unit of data of the replicated volume, wherein the source system wrote the write data to the source volume based upon the write request;   instructions to write the write data to the replicated volume;   instructions to update the data structure to indicate the write data has been written after the receipt of the start replication message;   instructions to receive source metadata associated with the source volume, wherein the metadata comprises an ordered list of block identifiers for data blocks of the source volume, and wherein each block identifier is used to access a data block;   instructions to compare the source metadata with prior metadata associated with a prior point-in-time image of the source volume to determine blocks of data that have changed since the prior point-in-time image of the source volume;   instructions to determine a first block of the blocks of data should not be retrieved based upon the data structure;   instructions to determine a second block of the blocks of data should be retrieved based upon the data structure;   instructions to retrieve the second block from the source system; and   instructions to write the second block to the replicated volume.   
     
     
         16 . The non-transitory computer-readable medium of  claim 15 , wherein the instructions further comprise instructions to send a replication complete message to the source system, wherein the source system no longer forwards I/O to the replica server based upon receipt of the replication complete message. 
     
     
         17 . The non-transitory computer-readable medium of  claim 15 , wherein the instructions further comprise instructions to:
 instructions to request verification data from the source system;   instructions to receive the verification data from the source system; and   instructions to compare the received verification data with corresponding verification data of the replica volume to determine the replication was successful.   
     
     
         18 . The non-transitory computer-readable medium of  claim 15 , wherein a unit of data is a sub-block, wherein a block consists of a plurality of sub-blocks, and wherein the instructions to write the second block comprise:
 instructions to determine which sub-blocks have been updated since the receipt of the start replication message based upon the data structure; and   instructions to write sub-blocks of the second block based upon on the sub-block not being updated since the receipt of the start replication message.   
     
     
         19 . The non-transitory computer-readable medium of  claim 15 , wherein the instructions further comprise instructions to:
 instructions to request quality of service parameters for the replicated volume;   instructions to request quality of service parameters for the source volume;   instructions to determine if the replication of the source volume to the replicated volume will succeed based upon the quality of service parameters of the replicated volume and the source volume.   
     
     
         20 . The non-transitory computer-readable medium of  claim 15 , wherein the data blocks of the replicated volume are randomly and evenly distributed across a cluster containing the replicated volume.