Flexible remote data mirroring

Abstract

Methods, systems, and configured storage media are provided for flexible data mirroring. In particular, the invention provides many-to-one data mirroring, including mirroring from local servers running the same or different operating systems and/or file systems at two or more geographically dispersed locations. The invention also provides one-to-many data mirroring, mirroring with or without a dedicated private telecommunications link, and mirroring with or without a dedicated server or another server at the destination(s) to assist the remote mirroring unit(s). In addition, the invention provides flexibility by permitting the use of various combinations of one or more external storage units and/or RAID units to hold mirrored data. Spoofing, SCSI and other bus emulations, and further tools and techniques are used in various embodiments of the invention.

1 . A method for storing data from a host comprising: 
 receiving a plurality of blocks of the data through a storage subsystem bus; and    storing each of the plurality of blocks of the data with a value indicating an order the block of the data was received.    
     
     
         2 . The method of  claim 1  wherein the storage subsystem bus is a standard storage subsystem bus.  
     
     
         3 . The method of  claim 1  wherein receiving the plurality of blocks of the data through the standard storage subsystem bus comprises: 
 emulating an interface of the standard storage subsystem bus;    receiving bus data from the emulated interface; and    extracting the plurality of blocks of the data from the bus data.    
     
     
         4 . The method of  claim 1  wherein storing each of the plurality of blocks of the data with the value indicating the order the portion of the data was received comprises: 
 creating a plurality of meta-data blocks, one for each of the plurality of blocks of the data, the meta-data blocks having the value indicating the order the corresponding block of the data was received; and    storing each meta-data block and the corresponding block of the data in a local buffer.    
     
     
         5 . The method of  claim 4  further comprising: 
 sending a pre-acknowledgement to the host after storing each meta-data block and the corresponding block of the data in the local buffer.    
     
     
         6 . The method of  claim 4  further comprising: 
 storing the plurality of blocks of the data in a local mirror; and    storing the plurality of blocks of the data to a remote mirror.    
     
     
         7 . The method of  claim 6  further comprising: 
 creating a snapshot of the data from the host from a first previous time in the remote mirror;    storing the plurality of blocks of the data received after the first previous time in a journal; and    recreating the data from the host from a second previous time, the second previous time is after the first previous time, using the snapshot and a subset of the data in the journal.    
     
     
         8 . The method of  claim 6  wherein storing the plurality of blocks of the data in a remote mirror comprises: 
 transmitting each of the plurality of blocks of the data and corresponding meta-data block through a journey link to a remote mirroring unit;    storing each of the plurality of blocks of the data and the corresponding meta-data block in a remote buffer; and    transmitting a plurality of remote acknowledgements over the journey link to the local mirroring unit, each of the plurality of remote acknowledgement corresponding to one of the plurality of blocks of the data, each remote acknowledgement transmitted after storing the corresponding block of the data in the remote buffer, each remote acknowledgement including a signature corresponding to the portion of the data associated with the remote acknowledgement.    
     
     
         9 . The method of  claim 8  further comprising: 
 verifying the signature of each of the plurality of remote acknowledgements;    transmitting a plurality of local acknowledgements to the remote mirroring unit over the journey link, each local acknowledgement corresponding to each local acknowledgement transmitted if the signature of the corresponding remote acknowledgement was verified; and    transmitting a plurality of local negative acknowledgements to the remote mirroring unit over the journey link, each local negative acknowledgement transmitted if the signature of the corresponding remote acknowledgement was not verified.    
     
     
         10 . The method of  claim 9  further comprising: 
 deleting a subset of the plurality of the blocks of the data and the corresponding meta-data blocks from the local buffer, each of the subset corresponding to the portion of the data corresponding to the remote acknowledgement with the signature that was verified.    
     
     
         11 . The method of  claim 8  further comprising: 
 sensing a state of connection or a state of disconnection of the journey link;    pausing transmission from the local mirroring unit to the remote mirroring unit and from the remote mirroring unit to the local mirroring unit when the journey link has the state of disconnection; and    resuming transmission from the local mirroring unit to the remote mirroring unit and from the remote mirroring unit to the local mirroring unit when the journey link has the state of connection after having the state of disconnection.    
     
     
         12 . The method of  claim 8  further comprising: 
 sensing a state of high traffic or a state of not high traffic on the journey link;    pausing transmission from the local mirroring unit to the remote mirroring unit and from the remote mirroring unit to the local mirroring unit when the journey link has the state of high traffic;    resuming transmission from the local mirroring unit to the remote mirroring unit and from the remote mirroring unit to the local mirroring unit when the journey link has the state of not high traffic after having the state of high traffic.    
     
     
         13 . The method of  claim 8  further comprising: 
 storing each of the plurality of the blocks of the data in a remote storage unit in the order indicated by the corresponding meta-data block.    
     
     
         14 . The method of  claim 13  further comprising: 
 recovering a previous state of the data from the host using the plurality of the blocks of the data in the remote storage unit, the plurality of the blocks of the data in the remote buffer, and the plurality of meta-data blocks associated with the plurality of the blocks of the data in the remote buffer.    
     
     
         15 . A system for storing data from a host comprising: 
 a local buffer having a first queue, structured to store the data from the host in the first queue;    a remote buffer having a second queue, structured to store the data from the first queue in the second queue; and    a remote storage unit structured to store the data from the second queue and to maintain an internally consistent copy of the data from the host.    
     
     
         16 . The system of  claim 15  wherein the local buffer is structured to acknowledge storing the data from the host to the host before receiving an acknowledgement from the remote buffer.  
     
     
         17 . The system of  claim 16  wherein the local buffer is structured to delete the data from the host after receiving an acknowledgement that the data stored in the remote buffer has been stored in the remote storage unit.  
     
     
         18 . The system of  claim 15  wherein the remote buffer is structured to generate an acknowledgement to acknowledge storing the data from the local buffer and send the acknowledgement to the local buffer.  
     
     
         19 . The system of  claim 18  wherein the remote buffer is structured to generate a signature of the data from the local buffer and send the signature with the acknowledgement.  
     
     
         20 . The system of  claim 19  wherein the local buffer is structured to verify the signature and send a second acknowledgement or a negative acknowledgement based on the verification of the signature.  
     
     
         21 . The system of  claim 20  wherein the remote buffer is structured to store the data from the local buffer in the remote storage unit after receiving the second acknowledgement.  
     
     
         22 . The system of  claim 21  wherein the local buffer is structured to delete the data from the host from the first queue after receiving a third acknowledgement acknowledging the storage of the data from the local buffer in the remote storage unit.  
     
     
         23 . The system of  claim 15  wherein the local buffer is structured to add a timestamp to the data from the host before storing the data from the host.  
     
     
         24 . The system of  claim 15  wherein the first queue and the second queue are each a first-in first-out (FIFO) queue.  
     
     
         25 . The system of  claim 15  further comprising: 
 a local mirror structured to store the data from the host; and    a local mirroring unit coupled to the local mirror, containing the local buffer.    
     
     
         26 . The system of  claim 15  wherein the system is a primary storage for the host.  
     
     
         27 . The system of  claim 15  wherein further comprising: 
 an interface structured to emulate a standard storage system bus, receive the data from the host and send the data from the host to the local buffer    
     
     
         28 . A system for storing data from a host comprising: 
 a journey link;    a local buffer coupled to the journey link, structured to transmit the data from the host over the journey link and pause the transmission if the journey link is interrupted;    a remote buffer coupled to the journey link, structured to store the data from the local buffer; and    a remote storage unit coupled to the remote buffer, structured to store the data from the remote buffer.    
     
     
         29 . The system of  claim 28  wherein the local buffer is structured to buffer data from the host received during a period during which the transmission is paused.  
     
     
         30 . The system of  claim 29  wherein the local buffer is structured to resume the transmission after pausing the transmission if the journey link is no longer interrupted.  
     
     
         31 . The system of  claim 28  further comprising: 
 a local mirror structured to store the data from the host; and    a local mirroring unit coupled to the local mirror, containing the local buffer.    
     
     
         32 . The system of  claim 28  wherein the system is the primary storage for the host.  
     
     
         33 . The system of  claim 28  wherein further comprising: 
 an interface structured to emulate a standard storage system bus, receive the data from the host and send the data from the host to the local buffer    
     
     
         34 . A system for storing data from a host comprising: 
 a local buffer structured to store the data from the host;    a remote buffer structured to store the data from the local buffer; and    a snapshot of the data from the host coupled to at least one of the group consisting of the local buffer and the remote buffer, the snapshot structured to store a version of the data from the host from a first previous time; and    a journal structured to store a plurality of data changes, each data change structured to store an incremental change to the data from the host.    
     
     
         35 . The system of  claim 34  wherein the local buffer is structured to recreate a state of the data from the host from a second previous time, the second previous time after the first previous time.  
     
     
         36 . The system of  claim 34  wherein the journal is a first-in first-out (FIFO) buffer structured to update the snapshot with the oldest entry in the journal and delete the oldest entry.  
     
     
         37 . The system of  claim 34  further comprising: 
 a local mirror structured to store the data from the host; and    a local mirroring unit coupled to the local mirror, containing the local buffer.    
     
     
         38 . The system of  claim 34  wherein the system is a primary storage for the host.  
     
     
         39 . The system of  claim 34  wherein further comprising: 
 an interface structured to emulate a standard storage system bus, receive the data from the host and send the data from the host to the local buffer    
     
     
         40 . A system for storing data from a host comprising: 
 a local buffer structured to store data from the host;    a plurality of remote buffers, each remote buffer structured to store the data from the local buffer; and    a plurality of remote storage units, each remote storage unit coupled one of the plurality of remote buffers and structured to store the data from the one of the plurality of remote buffers.    
     
     
         41 . The system of  claim 40  further comprising: 
 a local mirror structured to store the data from the host; and    a local mirroring unit coupled to the local mirror, containing the local buffer.    
     
     
         42 . The system of  claim 40  wherein the system is a primary storage for the host.  
     
     
         43 . The system of  claim 40  wherein further comprising: 
 an interface structured to emulate a standard storage system bus, receive the data from the host and send the data from the host to the local buffer.