Flexible remote data mirroring

Abstract

Methods, systems, and configured storage media are provided for flexible data mirroring. In particular, the invention provides many-to-one data mirroring, including mirroring from local servers running the same or different operating systems and/or file systems at two or more geographically dispersed locations. The invention also provides one-to-many data mirroring, mirroring with or without a dedicated private telecommunications link, and mirroring with or without a dedicated server or another server at the destination(s) to assist the remote mirroring unit(s). In addition, the invention provides flexibility by permitting the use of various combinations of one or more external storage units and/or RAID units to hold mirrored data. Spoofing, SCSI and other bus emulations, and further tools and techniques are used in various embodiments of the invention.

What is claimed and desired to be secured by patent is: 
     
       1. A method, for storing data, performed by a host computing system, the method comprising:
 obtaining an initial state of a storage volume; 
 transmitting an indication of the initial state of the storage volume; 
 generating a plurality of blocks of the data, the plurality of blocks of the data representing a change from a first state of the storage volume to a second state of the storage volume, wherein each of the plurality of blocks of the data is associated with a value indicating an order, and wherein each value associated with a data block corresponds to a sequence in which the host computing system stores the data block to a first persistent store; 
 transmitting, to a mirroring unit operationally independent from the host computing system, the plurality of blocks of the data configured in a manner suitable for the mirroring unit to (A) determine that a second persistent store is available for a write operation, and, in response to the determining, to (B) store each of the plurality of blocks of the data to the second persistent store; 
 transmitting an indication of a particular location in the sequence, the indication configured in a manner suitable for the mirroring unit to, in response to receiving the indication of the particular location in the sequence: 
 retrieve, from the second persistent store, based on the associated values that correspond to the sequence, one or more distinguished blocks of the data of the plurality of blocks of the data, the distinguished blocks comprising less than all of the plurality of blocks of the data and selected based on the particular location in the sequence; and 
 create, by applying the distinguished blocks of the data to the initial state of the storage volume, a structure representing a third state of the storage volume; and 
 receiving the structure representing the third state of the storage volume, wherein the third state of the storage volume is a state the storage volume was in after the storage volume was in the first state but prior to the storage volume being in the second state. 
 
     
     
       2. The method of  claim 1  wherein the plurality of blocks of the data are transmitted through a storage subsystem bus, and wherein the transmitting the plurality of blocks of the data through the standard storage subsystem bus comprises:
 emulating an interface of the standard storage subsystem bus; and 
 transmitting bus data to the emulated interface. 
 
     
     
       3. The method of  claim 1  wherein the associated values indicating an order are stored in a plurality of meta-data blocks, each meta-data block associated with one of the blocks of the data. 
     
     
       4. The method of  claim 1  further comprising:
 sensing an interruption or a state of high traffic in a connection to the mirroring unit operationally independent from the host computing system; and 
 in response to the sensing, pausing transmission of the plurality of blocks of the data to the mirroring unit operationally independent from the host computing system. 
 
     
     
       5. The method of  claim 4  further comprising:
 a local buffer configured to buffer data from the host computing system while the transmission is paused. 
 
     
     
       6. The method of  claim 4  further comprising:
 detecting a reconnection or a reduction in traffic in the connection to the mirroring unit operationally independent from the host computing system; and 
 in response to the detecting, resuming transmission of the plurality of blocks of the data to the mirroring unit operationally independent from the host computing system. 
 
     
     
       7. The method of  claim 1  wherein the initial state of the storage volume is a snapshot of the data from a first previous time; and
 wherein the mirroring unit operationally independent from the host computing system is configured to:
 store the plurality of blocks of the data in a journal; and 
 create the structure representing the third state of the storage volume by recreating the data from the host computing system, from a second previous time after the first previous time, using the snapshot and a subset of the data in the journal. 
 
 
     
     
       8. The method of  claim 1  further comprising:
 receiving one or more acknowledgement messages, wherein each acknowledgement message indicates that one or more of the plurality of blocks of the data has been stored to the second persistent store. 
 
     
     
       9. The method of  claim 8  wherein each acknowledgement message includes a signature corresponding to the blocks of the data that have been acknowledged as stored to the second persistent store. 
     
     
       10. The method of  claim 9  further comprising:
 determining that an attempt to verify at least one of the signatures has failed; and 
 transmitting a negative acknowledgement to the mirroring unit indicating a corresponding acknowledgement verification failure. 
 
     
     
       11. The method of  claim 1  wherein the value indicating an order is a timestamp. 
     
     
       12. A computer-readable device storing instructions configured to, when executed by a host computing system, cause the host computing system to perform operations for backing up data, the operations comprising:
 obtaining an initial state of a storage volume; 
 transmitting an indication of the initial state of the storage volume; 
 generating a plurality of blocks of the data, the plurality of blocks of the data representing a change from a first state of the storage volume to a second state of the storage volume, wherein each of the plurality of blocks of the data is associated with a value indicating an order, and wherein each value associated with a data block corresponds to a sequence in which the host computing system stores the data block to a first persistent store; 
 transmitting, to a mirroring unit operationally independent from the host computing system, the plurality of blocks of the data, wherein the mirroring unit is configured (A) to determine that a second persistent store is available for a write operation and (B) in response to the determining, to store each of the plurality of blocks of the data to the second persistent store; 
 transmitting an indication of a particular location in the sequence, wherein the mirroring unit is further configured to, in response to receiving the indication of the particular location in the sequence: 
 retrieve, from the second persistent store, based on the associated values that correspond to the sequence, one or more distinguished blocks of the data of the plurality of blocks of the data, the distinguished blocks comprising less than all of the plurality of blocks of the data and selected based on the particular location in the sequence; and 
 create, by applying the distinguished blocks of the data to the initial state of the storage volume, a structure representing a third state of the storage volume; and 
 receiving the structure representing the third state of the storage volume, wherein the third state of the storage volume is a state the storage volume was in after the storage volume was in the first state but prior to the storage volume being in the second state. 
 
     
     
       13. The computer-readable device of  claim 12  wherein the associated values indicating an order are stored in a plurality of meta-data blocks, each meta-data block associated with one of the blocks of the data. 
     
     
       14. The computer-readable device of  claim 12  wherein the initial state of the storage volume is a snapshot of the data from a first previous time; and
 wherein the mirroring unit operationally independent from the host computing system is configured to: 
 store the plurality of blocks of the data in a journal; and 
 create the structure representing the third state of the storage volume by recreating the data from the host computing system, from a second previous time after the first previous time, using the snapshot and a subset of the data in the journal. 
 
     
     
       15. The computer-readable device of  claim 12 , wherein the operations further comprise:
 receiving one or more acknowledgement messages, wherein each acknowledgement message indicates that one or more of the plurality of blocks of the data has been stored to the second persistent store. 
 
     
     
       16. The computer-readable device of  claim 15  wherein each acknowledgement message includes a signature corresponding to the blocks of the data that have been acknowledged as stored to the second persistent store. 
     
     
       17. The computer-readable device of  claim 16 , wherein the operations further comprise:
 determining that an attempt to verify at least one of the signatures has failed; and 
 transmitting a negative acknowledgement to the mirroring unit indicating a corresponding acknowledgement verification failure. 
 
     
     
       18. A host computing system configured to perform operations for backing up data, the system comprising:
 one or more processors; 
 a memory; 
 a storage volume configured to: 
 provide an initial state of the storage volume; and 
 generate a plurality of blocks of the data, the plurality of blocks of the data representing a change from a first state of the storage volume to a second state of the storage volume, wherein each of the plurality of blocks of the data is associated with a value indicating an order, and wherein each value associated with a data block corresponds to a sequence in which the host computing system stores the data block to a first persistent store; and 
 a transceiver configured to: 
 transmit an indication of the initial state of the storage volume; 
 transmit, to a mirroring unit operationally independent from the host computing system, the plurality of blocks of the data, wherein the mirroring unit is configured (A) to determine that a second persistent store is available for a write operation and (B) in response to the determining, to store each of the plurality of blocks of the data to the second persistent store; 
 transmit an indication of a particular location in the sequence, wherein the mirroring unit is further configured to, in response to receiving the indication of the particular location in the sequence: 
 retrieve, from the second persistent store, based on the associated values that correspond to the sequence, one or more distinguished blocks of the data of the plurality of blocks of the data, the distinguished blocks comprising less than all of the plurality of blocks of the data and selected based on the particular location in the sequence; and 
 create, by applying the distinguished blocks of the data to the initial state of the storage volume, a structure representing a third state of the storage volume; and 
 receive the structure representing the third state of the storage volume, wherein the third state of the storage volume is a state the storage volume was in after the storage volume was in the first state but prior to the storage volume being in the second state. 
 
     
     
       19. The system of  claim 18 , wherein the transceiver is further configured to:
 receive one or more acknowledgement messages, wherein each acknowledgement message indicates that one or more of the plurality of blocks of the data has been stored to the second persistent store; and 
 wherein each acknowledgement message includes a signature corresponding to the blocks of the data that have been acknowledged as stored to the second persistent store. 
 
     
     
       20. The system of  claim 19 , wherein the transceiver is further configured to, in response to determining that an attempt to verify at least one of the signatures has failed, transmit a negative acknowledgement to the mirroring unit indicating a corresponding acknowledgement verification failure.