Log driven storage controller with network persistent memory

Abstract

A method of updating database volumes by writing database updates to a smart storage controller, and having the smart storage controller generate the log update records for updating local and remote log volumes, and data volumes.

What is claimed is: 
     
       1. A storage controller, comprising:
 a processor; 
 a Random Access Memory (RAM) coupled to the processor, at least a portion of the RAM configured to be non-volatile, and a portion of the RAM utilized as a staging area for read-modify-write operations to a database; 
 a first network interface coupled to the memory and configured to couple to a communication network, the first network interface configured to perform remote direct memory access (RDMA) to the RAM based on instructions received over the communication network; and 
 a second network interface coupled to the processor and the RAM, the second network interface different than the first network interface, and the second network interface configured to read and write one or more memory disks; 
 the storage controller is configured to:
 accept a first RDMA write of a log record for a first database transaction across the first network interface; and 
 accept a second RDMA write of a log record for a second database transaction, different than the first database transaction, across the first network interface; and 
 
 the processor is configured to:
 write a first set of log and data volumes regarding the first database transaction across the second network interface, contents of the write based only on the first RDMA write of the log record; 
 determine contents of a write to the data volume for the second database transaction based only on the contents of the second RDMA write of the log record; and 
 write a block of data to the data volume, the block of data containing both the content of the first database transaction and the second database transaction; 
 wherein each write to a log or data volume for the first database transaction and the second database transaction is performed off the commit path of the database. 
 
 
     
     
       2. The storage controller of  claim 1  wherein the processor is further configured to write a second set of log and data volumes, different that the first set, based only on the first RDMA write of the log record. 
     
     
       3. The storage controller of  claim 1  wherein the write to the data volume is not block aligned. 
     
     
       4. The storage controller of  claim 1 , wherein the processor is configured to identify in the log record of the first RDMA write, new values to be written to the data volumes, and extract the new values from the log record of the first RDMA write. 
     
     
       5. The storage controller of  claim 4 , wherein the RAM is divided into a persistent memory region and buffer cache region. 
     
     
       6. The storage controller of  claim 5 , wherein the processor is configured to transfer the new values to the buffer cache region to update a block retrieved from one of the data volumes and stored in the buffer cache region. 
     
     
       7. The storage controller of  claim 6 , wherein the processor is configured to initiate the update and write the updated block to the data volume responsive only to the first RDMA write of the log record. 
     
     
       8. A method, comprising:
 performing, by a first network interface of a storage controller, remote direct memory access (RDMA) to a random access memory (RAM) based on instructions received by the first network interface over a communication network, wherein at least a portion of the RAM configured to be non-volatile, and a portion of the RAM is utilized as a staging area for read-modify-write operations to a database; 
 reading, by a second network interface of the storage controller, from one or more memory disks wherein the second network interface different than the first network interface; 
 writing, by the second network interface of the storage controller, to one or more memory disks; 
 receiving, by the storage controller, a first RDMA write of a log record for a first database transaction across the first network interface; 
 writing, by the storage controller, to a first set of log and data volumes regarding the first database transaction across the second network interface, wherein contents of the write are based only on the first RDMA write of the log; 
 receiving, by the storage controller, a second RDMA write of a log record for a second database transaction across the first network interface, wherein the second database transaction is different from the first database transaction; 
 determining, by the storage controller, contents of a write to the data volume for the second database transaction based only on the contents of the second RDMA write of the log record; and
 writing, by the storage controller, a block of data to the data volume, wherein the block of data contains both the content of the first database transaction and the contents of the second database transaction; 
 
 wherein each write to a log or data volume for the first database transaction and the second database transaction is performed off the commit path of a database. 
 
     
     
       9. The method of  claim 8 , further comprising:
 writing, by the storage controller, a second set of log and data volumes, wherein the second set is different from the first set and is based only on the first RDMA write of the log record. 
 
     
     
       10. The method of  claim 8 , wherein the write to the data volume is not block aligned. 
     
     
       11. The method of  claim 8 , further comprising:
 dividing the RAM into a persistent memory region and buffer cache region; 
 storing the received first RDMA write of a log record in the persistent memory region. 
 
     
     
       12. The method of  claim 11 , further comprising:
 identifying in the log record of the first RDMA write, by the storage controller, new values to be written to the data volumes, and 
 extracting the new values from the log record of the first RDMA write. 
 
     
     
       13. The method of  claim 12 , further comprising transferring the new values to the buffer cache region to update a block retrieved from one of the data volumes and stored in the buffer cache region. 
     
     
       14. The method of  claim 13 , further comprising initiating the update and writing of the updated block to the data volume responsive only to the first RDMA write of the log record. 
     
     
       15. A non-transitory, storage medium containing computer software that when executed by a processor, causes a smart storage controller to:
 perform a remote direct memory access (RDMA) to a random access memory (RAM) based on instructions received by a first network interface over a communication network wherein at least a portion of the RAM configured to be non-volatile, and a portion of the RAM is utilized as a staging area for read-modify-write operations to a database;
 read from one or more memory disks using a second network interface; 
 write to one or more memory disks using the second network interface; 
 wherein the second network interface different than the first network interface; 
 receive a first RDMA write of a log record for a first database transaction across a first network interface; 
 write to a first set of log and data volumes regarding the first database transaction across a second network interface, wherein contents of the write are based only on the first RDMA write of the log; 
 receive a second RDMA write of a log record for a second database transaction, different than the first database transaction, across the first network interface; 
 determine contents of a write to the data volume for the second database transaction based only on the contents of the second RDMA write of the log record; and 
 write a block of data to the data volume, the block of data containing both the content of the first database transaction and the second database transaction; 
 
 wherein each write to a log or data volume for the first database transaction and the second database transaction is performed off the commit path of a database. 
 
     
     
       16. The non-transitory, storage medium containing computer software of  claim 15 , further causes a smart storage controller to:
 write a second set of log and data volumes, different that the first set, based only on the first RDMA write of the log record. 
 
     
     
       17. The non-transitory, storage medium containing computer software of  claim 15 , wherein the write to the data volume is not block aligned. 
     
     
       18. The non-transitory, storage medium containing computer software of  claim 15 , that further causes the smart storage controller to:
 divide the RAM into a persistent memory region and buffer cache region; 
 store the received first RDMA write of a log record in the persistent memory region; 
 Identify, in the log record of the first RDMA write, new values to be written to the data volumes, and 
 extract the new values from the log record of the first RDMA write. 
 
     
     
       19. The non-transitory, storage medium containing computer software of  claim 18 , that further causes the smart storage controller to transfer the new values to the buffer cache region to update a block retrieved from one of the data volumes and stored in the buffer cache region. 
     
     
       20. The non-transitory, storage medium containing computer software of  claim 19 , that further causes the smart storage controller to initiate the update and writing of the updated block to the data volume responsive only to the first RDMA write of the log record.