Policy-based hierarchical data protection in distributed storage

Abstract

A storage management computing device obtains an information lifecycle management (ILM) policy. A data protection scheme to be applied at a storage node computing device level is determined and a plurality of storage node computing devices are identified based on an application of the ILM policy to metadata received from one of the storage node computing devices and associated with an object ingested by the one of the storage node computing devices. The one of the storage node computing devices is instructed to generate one or more copies of the object or fragments of the object according to the data protection scheme and to distribute the object copies or one of the object fragments to one or more other of the storage node computing devices to be stored by at least the one or more other storage node computing devices on one or more disk storage devices.

What is claimed is: 
     
       1. A method comprising:
 parsing, by a first computing device, an information lifecycle management (ILM) policy that indicates a first data protection scheme to apply at a first level of a distributed storage system at object ingest based on object metadata and a second data protection scheme to apply at the first level of the distributed storage system based on a time parameter; 
 identifying an object, which is already stored in the distributed storage system according to the first data protection scheme, to which the information lifecycle management policy applies and for which the time parameter is satisfied; and 
 causing, by the first computing device, multiple storage nodes of the distributed storage system to store the object according to the second data protection scheme at the first level. 
 
     
     
       2. The method of  claim 1 , wherein the first computing device causes the object to be protected in the distributed storage system according to a third data protection scheme at a second level corresponding to storage devices of the multiple storage nodes. 
     
     
       3. The method of  claim 1 , wherein the first data protection scheme comprises an item selected from a list consisting of: a replication data protection scheme and an erasure coding data protection scheme. 
     
     
       4. The method of  claim 1 , wherein the second data protection scheme comprises an item selected from a list consisting of: a replication data protection scheme and an erasure coding data protection scheme. 
     
     
       5. The method of  claim 1 , wherein a third data protection scheme indicated in the information lifecycle management policy comprises an item selected from a list consisting of: a redundant array of independent disks (RAID), dynamic disk pool (DDP), and just a bunch of disks (JBOD) data protection scheme. 
     
     
       6. The method of  claim 1 , wherein identifying the object comprises determining a namespace indicated by the information lifecycle management policy and determining that the object is within the namespace. 
     
     
       7. The method of  claim 1 , further comprising the first computing device causing the object, as stored according to the first data protection scheme, to be removed from the distributed storage system. 
     
     
       8. A non-transitory machine readable medium having stored thereon instructions for performing a method comprising machine executable code which when executed by at least one machine, causes the at least one machine to:
 parse an information lifecycle management policy that indicates a first data protection scheme to apply at a first level of a distributed storage system at object ingest based on object metadata and a second data protection scheme to apply at the first level of the distributed storage system based on a time parameter; 
 identify an object to which the information lifecycle management policy applies based on metadata of the object, wherein the object has already been protected in the distributed storage system according to the first data protection scheme, 
 determine that the time parameter is satisfied, and 
 instruct multiple storage nodes of the distributed storage system to store the object according to the second data protection scheme at the first level. 
 
     
     
       9. The non-transitory machine readable medium of  claim 8 , wherein the code to instruct the multiple storage nodes causes the object to be protected in the distributed storage system according to a third data protection scheme at a second level corresponding to storage devices of the multiple storage nodes. 
     
     
       10. The non-transitory machine readable medium of  claim 8 , wherein identifying an object is based on evaluating the information lifecycle management policy. 
     
     
       11. The non-transitory machine readable medium of  claim 8 , wherein the first data protection scheme comprises an item selected from a list consisting of: a replication data protection scheme or an erasure coding data protection scheme. 
     
     
       12. The non-transitory machine readable medium of  claim 8 , wherein a third data protection scheme indicated in the information lifecycle management policy comprises an item selected from a list consisting of: a redundant array of independent disks (RAID), dynamic disk pool (DDP), or just a bunch of disks (JBOD) data protection scheme. 
     
     
       13. The non-transitory machine readable medium of  claim 8 , wherein the code to identify the object comprises code to determine a namespace indicated by the information lifecycle management policy. 
     
     
       14. The non-transitory machine readable medium of  claim 8 , wherein the code to instruct the storage nodes comprises code to remove the object, as stored according to the first data protection scheme. 
     
     
       15. A computing device comprising:
 a memory containing machine readable medium comprising machine executable code having stored thereon instructions for performing a method of applying data protection; 
 a processor coupled to the memory, the processor configured to execute the machine executable code to cause the processor to: 
 parse, by a first computing device, an information lifecycle management (ILM) policy that indicates a first data protection scheme to apply at a first level of a distributed storage system at object ingest based on object metadata and a second data protection scheme to apply at the first level of the distributed storage system based on a time parameter; 
 identify an object, which is already stored in the distributed storage system according to the first data protection scheme, to which the information lifecycle management policy applies and for which the time parameter is satisfied; and 
 cause, by the first computing device, multiple storage nodes of the distributed storage system to store the object according to the second data protection scheme at the first level. 
 
     
     
       16. The computing device of  claim 15 , wherein the first data protection scheme comprises an item selected from a list consisting of: a replication data protection scheme and an erasure coding data protection scheme, and wherein a third data protection scheme indicated in the information lifecycle management policy comprises an item selected from a list consisting of a redundant array of independent disks (RAID), dynamic disk pool (DDP), and just a bunch of disks (JBOD) data protection scheme. 
     
     
       17. The computing device of  claim 15 , wherein the code to cause the multiple storage nodes to store the object comprises code to instruct the multiple storage nodes of the distributed storage system to store the object. 
     
     
       18. The computing device of  claim 15 , wherein the code to identify the object comprises code to determine a namespace indicated by the information lifecycle management policy. 
     
     
       19. The computing device of  claim 15 , wherein the machine readable medium further comprises code to periodically evaluate a plurality of other information lifecycle management policies. 
     
     
       20. The computing device of  claim 15 , wherein the code to parse the information lifecycle management policy comprises code to evaluate the information lifecycle management policy in response to retrieval of an object to which the information lifecycle management policy applies.