Incremental vault to object store

Abstract

A method includes receives data representing a changed chunk of data in a current revision of a data volume, the changed chunk includes data having changes from previous data of a previous revision of the data volume. The method creates a block of data representing the changed chunk of data on the object store, the object store also stores previous revision data of the previous revision. The method determines a previous index stored on the object store corresponding to the previous revision, which includes entries including at least one corresponding to the previous revision data. The method creates a revised index that updates the corresponding entry with updated entry data representing the changed chunk of data. The method includes deleting, from the object store, each particular block of data stored on the object store that is no longer associated with an entry on any index stored on the object store.

What is claimed is: 
     
       1. A computer-implemented method executed by data processing hardware that causes the data processing hardware to perform operations comprising:
 receiving first data representing a changed chunk of data in a revision of a data volume stored on an object store, the changed chunk of data comprising data having changes from previous data of a previous revision of the data volume; 
 generating a changed block of data representing a copy of the changed chunk of data for storage on the object store as object data, wherein the object store further stores a previous revision block of data representing previous revision data of the data volume stored as object data; 
 determining a previous index stored on the object store corresponding to the previous revision of the data volume, the previous index comprising at least one index page having entries, at least one entry corresponding to the previous revision block of data stored on the object store; 
 generating a revised index corresponding to a current revision of the data volume, wherein creating the revised index comprises:
 generating a copy of the previous index; 
 determining at least one entry in the copy corresponding to the previous revision block of data; and 
 updating the at least one entry with updated entry data representing the changed block of data to create the revised index, such that the current revision of the data volume is retrievable from the object store using the revised index; and 
 
 deleting, from the object store, each particular block of data stored on the object store that is no longer associated with an entry on any index stored on the object store. 
 
     
     
       2. The computer-implemented method of  claim 1 , wherein the operations further comprise storing the revised index on the object store. 
     
     
       3. The computer-implemented method of  claim 1 , wherein the operations further comprise retrieving for storage on a storage device, using the index, the current revision of the data volume. 
     
     
       4. The computer-implemented method of  claim 1 , wherein index entries in the revised index corresponding to chunks of data in the current revision of the data volume that are unchanged from the previous revision of the data volume are the same as corresponding index entries in the previous index. 
     
     
       5. The computer-implemented method of  claim 1 , wherein the operations further comprise:
 receiving a request to restore the previous revision of the data volume stored on the object store; 
 retrieving the revised index corresponding to the current revision of the data volume and the previous index corresponding to the previous revision of the data volume; 
 identifying a set of changed blocks stored on the object store corresponding to entries in the revised index, the set of changed blocks being blocks of data representing data that have changed from the previous revision of the data volume; and 
 retrieving the set of changed blocks. 
 
     
     
       6. The computer-implemented method of  claim 5 , wherein the set of changed blocks correspond to index entries in the revised index that differ from corresponding index entries in the previous index. 
     
     
       7. The computer-implemented method of  claim 5 , wherein the operations further comprise copying the set of changed blocks to a storage device thereby restoring the previous revision of the data volume to the storage device. 
     
     
       8. The computer-implemented method of  claim 5 , wherein the operations further comprise mounting requested blocks of the set of changed blocks such that they are accessible by a storage device, thereby accomplishing an on demand restore of the previous revision of the data volume to the storage device. 
     
     
       9. The computer-implemented method of  claim 1 , wherein the operations further comprise:
 receiving a request to calculate a space reclamation amount associated with deleting the previous revision of the data volume stored on the object store; 
 retrieving the revised index and the previous index; 
 determining deletable blocks of data by comparing index entries in the revised index to corresponding index entries in the previous index, the deletable blocks of data having corresponding index entries in the revised index that differ from corresponding data entries in the previous index; 
 calculating a size of each of the deletable blocks of data; 
 calculating a size of metadata corresponding to the deletable blocks of data; and 
 calculating the space reclamation amount as a sum of the sizes of the deletable blocks of data and the metadata. 
 
     
     
       10. The computer-implemented method of  claim 1 , wherein generating the changed block of data comprises compressing the changed chunk of data. 
     
     
       11. A system comprising:
 data processing hardware; and 
 memory hardware in communication with the data processing hardware and storing instructions that, when executed on the data processing hardware, causes the data processing hardware to perform operations comprising:
 receiving first data representing a changed chunk of data in a revision of a data volume stored on an object store, the changed chunk of data comprising data having changes from previous data of a previous revision of the data volume; 
 generating a changed block of data representing a copy of the changed chunk of data for storage on the object store as object data, wherein the object store further stores a previous revision block of data representing previous revision data of the data volume stored as object data; 
 determining a previous index stored on the object store corresponding to the previous revision of the data volume, the previous index comprising at least one index page having entries, at least one entry corresponding to the previous revision block of data stored on the object store; 
 generating a revised index corresponding to a current revision of the data volume, wherein creating the revised index comprises:
 generating a copy of the previous index; 
 determining at least one entry in the copy corresponding to the previous revision block of data; and 
 updating the at least one entry with updated entry data representing the changed block of data to create the revised index, such that the current revision of the data volume is retrievable from the object store using the revised index; and 
 
 deleting, from the object store, each particular block of data stored on the object store that is no longer associated with an entry on any index stored on the object store. 
 
 
     
     
       12. The system of  claim 11 , wherein the operations further comprise storing the revised index on the object store. 
     
     
       13. The system of  claim 11 , wherein the operations further comprise retrieving for storage on a storage device, using the index, the current revision of the data volume. 
     
     
       14. The system of  claim 11 , wherein index entries in the revised index corresponding to chunks of data in the current revision of the data volume that are unchanged from the previous revision of the data volume are the same as corresponding index entries in the previous index. 
     
     
       15. The system of  claim 11 , wherein the operations further comprise:
 receiving a request to restore the previous revision of the data volume stored on the object store; 
 retrieving the revised index corresponding to the current revision of the data volume and the previous index corresponding to the previous revision of the data volume; 
 identifying a set of changed blocks stored on the object store corresponding to entries in the revised index, the set of changed blocks being blocks of data representing data that have changed from the previous revision of the data volume; and 
 retrieving the set of changed blocks. 
 
     
     
       16. The system of  claim 15 , wherein the set of changed blocks correspond to index entries in the revised index that differ from corresponding index entries in the previous index. 
     
     
       17. The system of  claim 15 , wherein the operations further comprise copying the set of changed blocks to a storage device thereby restoring the previous revision of the data volume to the storage device. 
     
     
       18. The system of  claim 15 , wherein the operations further comprise mounting requested blocks of the set of changed blocks such that they are accessible by a storage device, thereby accomplishing an on demand restore of the previous revision of the data volume to the storage device. 
     
     
       19. The system of  claim 11 , wherein the operations further comprise:
 receiving a request to calculate a space reclamation amount associated with deleting the previous revision of the data volume stored on the object store; 
 retrieving the revised index and the previous index; 
 determining deletable blocks of data by comparing index entries in the revised index to corresponding index entries in the previous index, the deletable blocks of data having corresponding index entries in the revised index that differ from corresponding data entries in the previous index; 
 calculating a size of each of the deletable blocks of data; 
 calculating a size of metadata corresponding to the deletable blocks of data; and 
 calculating the space reclamation amount as a sum of the sizes of the deletable blocks of data and the metadata. 
 
     
     
       20. The system of  claim 11 , wherein generating the changed block of data comprises compressing the changed chunk of data.