Single input/output writes in a file system hosted on a cloud, virtual, or commodity-server platform

Abstract

Systems and methods for performing single I/O writes are provided. According to one embodiment, responsive to receipt of a write operation from a client by a file system layer of a node of a distributed storage system and a data payload of the operation having been determined to meet a compressibility threshold, an intermediate storage layer of the node logically interposed between the file system layer and a block storage media is caused to perform a single input/output (I/O) write operation that persists the compressed data payload and corresponding metadata to support asynchronous journaling of the write operation. The single I/O write operation coupled with the use of a new pool file that maintains a list of available blocks for single I/O write operations and a modified node crash recovery approach allows the write operation to be acknowledged to the client while the journaling is performed asynchronously.

What is claimed is: 
     
       1. A non-transitory machine readable medium storing instructions, which when executed by a processing resource of a node of distributed storage system, cause the node to:
 responsive to receiving a write operation from a client by a file system layer of the node and determining a data payload of the write operation meets a compressibility threshold, cause an intermediate storage layer of the node logically interposed between the file system layer and a block storage media to perform a single input/output (I/O) write operation, wherein the single I/O write operation involves writing a packed block header containing an operation header entry corresponding to the write operation, and the data payload in compressed form to a data block associated with a particular block number within the block storage media; and 
 responsive to completion of the single I/O write:
 initiate, by the file system layer, journaling of an operation header containing the particular block number; and 
 without waiting for completion of the journaling, acknowledge, by the file system layer, receipt of the write operation to the client. 
 
 
     
     
       2. The non-transitory machine readable medium of  claim 1 , wherein the instructions further cause the node to:
 maintain, by the file system layer, a persistent on-disk data structure containing information regarding a plurality of block numbers that are available for single I/O write operations; 
 mark the particular block number within the persistent on-disk data structure as being associated with a particular consistency point (CP) active at a time of the single I/O write operation; and 
 prior to performing the single I/O write operation, store information regarding the particular CP within metadata of the packed block header. 
 
     
     
       3. The non-transitory machine readable medium of  claim 2 , wherein the journaling includes logging to a journal and mirroring of the journal to a high-availability (HA) partner node of a second distributed storage system and wherein the instructions further cause the node to during recovery from a crash of the node, identify (i) those of a plurality of single I/O write operations performed by the node prior to performance of a last CP by the node that are to be reconstructed and replayed based on the persistent on-disk data structure, (ii) information regarding the last CP, and (iii) operation headers contained in the journal. 
     
     
       4. The non-transitory machine readable medium of  claim 3 , wherein identification of said those of a plurality of single I/O write operations comprises for any block numbers marked in the persistent on-disk data structure as being associated with the last CP, that are not present in the journal, determining whether a corresponding data block persisted to the collection of disks represents a valid single I/O data block based on existence of a packed block header within the corresponding data block. 
     
     
       5. The non-transitory machine readable medium of  claim 4 , wherein the instructions further cause the node to for each identified single I/O write operation of the identified plurality of single I/O write operations:
 reconstruct the single I/O write operation based on the metadata of the packed block header and the compressed data payload of the corresponding data block; and 
 replay the reconstructed single I/O write operation. 
 
     
     
       6. The non-transitory machine readable medium of  claim 1 , wherein the node comprises a virtual storage system or a commodity computer system without battery-backed non-volatile random access memory. 
     
     
       7. A method comprising:
 responsive to receiving a write operation from a client by a file system layer of a node of a distributed storage system and determining a data payload of the write operation meets a compressibility threshold, causing an intermediate storage layer of the node logically interposed between the file system layer and a block storage media to perform a single input/output (I/O) write operation, wherein the single I/O write operation involves writing a packed block header containing an operation header entry corresponding to the write operation, and the data payload in compressed form to a data block associated with a particular block number within the block storage media; and 
 responsive to completion of the single I/O write:
 initiating, by the file system layer, journaling of an operation header containing the particular block number; and 
 without waiting for completion of the journaling, acknowledging, by the file system layer, receipt of the write operation to the client. 
 
 
     
     
       8. The method of  claim 7 , further comprising:
 maintaining, by the file system layer, a data structure containing information regarding a plurality of block numbers that are available for single I/O write operations; 
 marking the particular block number within the data structure as being associated with a particular consistency point (CP) active at a time of the single I/O write operation; and 
 prior to performing the single I/O write operation, storing information regarding the particular CP within metadata of the packed block header. 
 
     
     
       9. The method of  claim 8 , wherein the journaling includes logging to a journal and mirroring of the journal to a high-availability (HA) partner node of a second distributed storage system and wherein the method further comprises during recovery from a crash of the node identifying those of a plurality of single I/O write operations performed by the node prior to performance of a last CP by the node that are to be reconstructed and replayed based on the data structure, information regarding the last CP, and operation headers contained in the journal. 
     
     
       10. The method of  claim 9 , wherein said identifying comprises for any block numbers marked in the data structure as being associated with the last CP, that are not present in the journal, determining whether a corresponding data block persisted to the collection of disks represents a valid single I/O data block based on existence of a packed block header within the corresponding data block. 
     
     
       11. The method of  claim 10 , further comprising for each identified single I/O write operation of the identified plurality of single I/O write operations:
 reconstructing the single I/O write operation based on the metadata of the packed block header and the compressed data payload of the corresponding data block; and 
 replaying the reconstructed single I/O write operation. 
 
     
     
       12. A distributed storage system comprising:
 a processing resource; and 
 a non-transitory computer-readable medium, coupled to the processing resource, having stored therein instructions that when executed by the processing resource cause a node of the distributed storage system to: 
 responsive to receiving a write operation from a client by a file system layer of the node and determining a data payload of the write operation meets a compressibility threshold, cause an intermediate storage layer of the node logically interposed between the file system layer and a block storage media to perform a single input/output (I/O) write operation, wherein the single I/O write operation involves writing a packed block header containing an operation header entry corresponding to the write operation, and the data payload in compressed form to a data block associated with a particular block number within the block storage media; and 
 responsive to completion of the single I/O write:
 initiate, by the file system layer, journaling of an operation header containing the particular block number; and 
 without waiting for completion of the journaling, acknowledge, by the file system layer, receipt of the write operation to the client. 
 
 
     
     
       13. The distributed storage system of  claim 12 , wherein the instructions further cause the node to:
 maintain, by the file system layer, a persistent on-disk data structure containing information regarding a plurality of block numbers that are available for single I/O write operations; 
 mark the particular block number within the persistent on-disk data structure as being associated with a particular consistency point (CP) active at a time of the single I/O write operation; and 
 prior to performing the single I/O write operation, store information regarding the particular CP within metadata of the packed block header. 
 
     
     
       14. The distributed storage system of  claim 13 , wherein the journaling includes logging to a journal and mirroring of the journal to a high-availability (HA) partner node of a second distributed storage system and wherein the instructions further cause the node to during recovery from a crash of the node, identify (i) those of a plurality of single I/O write operations performed by the node prior to performance of a last CP by the node that are to be reconstructed and replayed based on the persistent on-disk data structure, (ii) information regarding the last CP, and (iii) operation headers contained in the journal. 
     
     
       15. The distributed storage system of  claim 14 , wherein identification of said those of a plurality of single I/O write operations comprises for any block numbers marked in the persistent on-disk data structure as being associated with the last CP, that are not present in the journal, determining whether a corresponding data block persisted to the collection of disks represents a valid single I/O data block based on existence of a packed block header within the corresponding data block. 
     
     
       16. The distributed storage system of  claim 15 , wherein the instructions further cause the node to for each identified single I/O write operation of the identified plurality of single I/O write operations:
 reconstruct the single I/O write operation based on the metadata of the packed block header and the compressed data payload of the corresponding data block; and 
 replay the reconstructed single I/O write operation. 
 
     
     
       17. The distributed storage system of  claim 13 , wherein the node comprises a virtual storage system. 
     
     
       18. The distributed storage system of  claim 13 , wherein the node comprises a commodity computer system without battery-backed non-volatile random access memory. 
     
     
       19. The distributed storage system of  claim 12 , wherein the intermediate storage layer comprises a redundant array of independent disks (RAID) layer. 
     
     
       20. The distributed storage system of  claim 19 , wherein the block storage media comprises a collection of disks managed by the RAID layer.