Online incremental database dump

Abstract

Various embodiments may include taking an initial dump of a database table and tracking changes to the database table, with regard to but outside of transactions, since a most recent full or partial dump has been taken. A partial dump, in such embodiments, may be taken since the most recent full or partial dump and used to create a data structure which may be used to update only those database table rows modified since a last full or partial dump. As a result, the dumps of a database table will include a single full dump and may include one or more partial dumps. These full and partial dumps may then be instantiated to create or update an instance of the database table. In some embodiments, dumps may be taken directly to a mirrored instance of a database table.

1. A method comprising:
 creating a backup copy of a database table; 
 inserting a marker in a modified row journal of the database table; 
 within a database management system garbage collector process before the garbage collector process purges journaling data associated with transactions that have committed, rolled back, timed out, or have otherwise terminated:
 identifying modified rows in the database table, and 
 writing a row id of each modified row to the modified row journal. 
 
 
     
     
       2. The method of  claim 1 , further comprising:
 reading into a modified row listing the row ids starting at a first row in the modified row journal following a most recent marker in the modified row journal; 
 copying the modified rows identified in the modified row listing from the database table into a table modification data structure; and 
 inserting a marker in the modified row journal of the database table. 
 
     
     
       3. The method of  claim 2 , wherein before the copying of the modified rows from the database table into the table modification data structure is performed, the method includes:
 identifying duplicate row ids in the modified row listing and removing all but one of the duplicate row ids. 
 
     
     
       4. The method of  claim 2 , wherein if a row id does not exist in the database table, the row id is written to the table modification data structure to signify that the row has been deleted from database table. 
     
     
       5. The method of  claim 2 , further comprising:
 creating a second instance of the database table; 
 copying data from the backup copy to the second instance of the database table; and 
 updating rows in the second instance of the database table as a function of the table modification data structure. 
 
     
     
       6. The method of  claim 5 , further comprising:
 further updating rows in the second instance of the database table as a function of a second table modification data structure including only data of rows identified in the modified row journal as modified since a latest marker in the modified row journal. 
 
     
     
       7. A database management system comprising:
 a full dump process operable to generate a backup copy of a database table and to insert a marker in a modified row journal of the database table; 
 a modified row identification process operable to insert an identifier of a modified row in a modified row journal of the database table after one or more transactions have successfully committed one or more modifications to one or more rows in the database table; and 
 a partial dump process operable to:
 create a data structure including current values of rows included in the modified row journal after a most recent marker in the modified row journal, and 
 insert a marker in the modified row journal indicating the data structure has been created. 
 
 
     
     
       8. The database management system of  claim 7 , wherein the modified row identification process is operable to insert the identifier of a modified row when a row is created, updated, or deleted. 
     
     
       9. The database management system of  claim 7 , wherein the modified row identification process is operable prior to or as part of a garbage collection process that operates to remove journaling data associated with transactions that have committed, rolled back, timed out, or have otherwise terminated. 
     
     
       10. The database management system of  claim 7 , wherein the full dump process and the partial dump process are operable to insert markers to the modified row journal identifying a respective dump process target. 
     
     
       11. The database management system of  claim 10 , wherein a dump process target is an identifier of a period for which the respective full or partial dump covers. 
     
     
       12. The database management system of  claim 10 , wherein there are two or more dump process targets. 
     
     
       13. A computer-readable medium, within instructions thereon, which when executed, cause one or more computing devices to perform a method comprising:
 creating a backup copy of a database table; 
 inserting a marker in a modified row journal of the database table; 
 within a database management system transient journal garbage collector process before the garbage collector process purges a transient journal journaling data associated with transactions that have committed, rolled back, timed out, or have otherwise terminated:
 identifying modified rows in the database table, and 
 writing a row id of each modified row to the modified row journal. 
 
 
     
     
       14. The computer-readable medium of  claim 13 , with further instructions of the method, which when executed, cause the one or more computing devices to:
 read into a modified row listing the row ids starting at a first row in the modified row journal following a most recent marker in the modified row journal; 
 copy the modified rows identified in the modified row listing from the database table into a table modification data structure; and 
 insert a marker in the modified row journal of the database table. 
 
     
     
       15. The computer-readable medium of  claim 14 , wherein before the copying of the modified rows from the database table into the table modification data structure is performed, the instructions are further operable to:
 identify duplicate row ids in the modified row listing and removing all but one of the duplicate row ids. 
 
     
     
       16. The computer-readable medium of  claim 14 , wherein if a row id does not exist in the database table, the row id is written to the table modification data structure to signify that the row has been deleted from database table. 
     
     
       17. The computer-readable medium of  claim 14 , with further instructions of the method, which when executed, cause the one or more computing devices to:
 create a second instance of the database table; 
 copy data from the backup copy to the second instance of the database table; and 
 updating rows in the second instance of the database table as a function of the table modification data structure. 
 
     
     
       18. The computer-readable medium of  claim 17 , with further instructions of the method, which when executed, cause the one or more computing devices to:
 further update rows in the second instance of the database table as a function of a second table modification data structure including only data of rows identified in the modified row journal as modified since a latest marker in the modified row journal.