Method of using common storage of parity data for unique copy recording

Abstract

A disclosed method is performed at a fault-tolerant object-based storage system including M data storage entities, each is configured to store data on an object-basis. The method includes obtaining a request to store N copies of a data object and in response, storing the N copies of the data object across the M data storage entities, where the N copies are distributed across the M data storage entities. The method additionally includes generating a first parity object for a first subset of M copies of the N copies of the data object, where the first parity object is stored on a first parity storage entity separate from the M data storage entities. The method also includes generating a manifest linking the first parity object with one or more other subsets of M copies of the N copies of the data object.

The invention claimed is: 
     
       1. A method comprising:
 at a fault-tolerant object storage system including M data storage entities, each configured to store data on an object-basis, and one or more controllers: 
 obtaining a request to store N copies of a data object within the fault-tolerant object storage system; 
 storing the N copies of the data object across the M data storage entities in response to the request, wherein the N copies of the data object are distributed across the M data storage entities such that any two sequential copies of the N copies are stored on two separate storage entities of the M data storage entities; 
 generating a first parity object for a first subset of M copies of the N copies of the data object, wherein the first parity object is stored on a first parity storage entity separate from the M data storage entities; 
 generating a manifest linking the first parity object with one or more other subsets of M copies of the N copies of the data object; 
 obtaining a parity compression indicator, wherein the parity compression indicator is user configurable; and 
 generating and storing a second parity object for a second subset of M copies of the N copies of the data object when the parity compression indicator is set to off, wherein the second parity object has a same value as the first parity object. 
 
     
     
       2. The method of  claim 1 , further comprising:
 determining a remainder number of the N copies of the data object, wherein the remainder number is less than M; and 
 generating a remainder parity object for the remainder number of the N copies of the data object. 
 
     
     
       3. The method of  claim 1 , further comprising:
 generating a third parity object for the first subset of M copies of the N copies of the data object, wherein the third parity object is different from the first parity object; 
 linking, within the manifest, the third parity object with the one or more other subsets of M copies of the N copies of the data object; and 
 storing the third parity object on a second parity storage entity separate from the first parity storage entity. 
 
     
     
       4. The method of  claim 1 , wherein the data object represents a portion of raw media data. 
     
     
       5. The method of  claim 1 , wherein:
 the N copies of the data object are evenly distributed across the M data storage entities; and 
 the first parity object is generated based on the M copies of the data object in the first subset. 
 
     
     
       6. The method of  claim 1 , wherein the first parity object includes a repeat indicator. 
     
     
       7. The method of  claim 1  further comprising restoring a missing data object using a corresponding parity object associated with the missing data object and corresponding data objects stored in the fault-tolerant object storage system. 
     
     
       8. The method of  claim 1 , wherein obtaining the request to store the N copies of the data object within the fault-tolerant object storage system includes:
 receiving multiple requests to store data within the fault-tolerant object storage system; 
 identifying the N copies of the data object in the data associated with the multiple requests; and 
 generating the request to store the N copies of the data object within the fault-tolerant object storage system. 
 
     
     
       9. A fault-tolerant object-based storage system comprising:
 M data storage entities, each configured to store data on an object-basis; 
 a controller with control links to each of the plurality of M data storage entities, wherein the controller is configured to: 
 obtain a request to store N copies of a data object within the fault-tolerant object storage system; 
 store the N copies of the data object across the M data storage entities in response to the request, wherein the N copies of the data object are distributed across the M data storage entities such that any two sequential copies of the N copies are stored on two separate storage entities of the M data storage entities; 
 generate a first parity object for a first subset of M copies of the N copies of the data object, wherein the first parity object is stored on a first parity storage entity separate from the M data storage entities; 
 generate a manifest linking the first parity object with one or more other subsets of M copies of the N copies of the data object; 
 obtain a parity compression indicator, wherein the parity compression indicator is user configurable; and 
 generate and store a second parity object for a second subset of M copies of the N copies of the data object when the parity compression indicator is set to off, wherein the second parity object has a same value as the first parity object. 
 
     
     
       10. The fault-tolerant object-based storage system of  claim 9 , wherein the controller is further configured to:
 determine a remainder number of the N copies of the data object, wherein the remainder number of copies is less than M; and 
 generate a remainder parity object for the remainder number of the N copies of the data object. 
 
     
     
       11. The fault-tolerant object-based storage system of  claim 9 , wherein the controller is further configured to:
 generate a third parity object for the first subset of M copies of the N copies of the data object, wherein the third parity object is different from the first parity object; 
 link, within the manifest, the third parity object with one or more other subsets of M copies of the N copies of the data object; and 
 store the third parity object on a second parity storage entity separate from the first parity storage entity. 
 
     
     
       12. The fault-tolerant object-based storage system of  claim 9 , wherein the controller is further configured to restore a missing data object using a corresponding parity object associated with the missing data object and corresponding data objects stored in the fault-tolerant object storage system. 
     
     
       13. The fault-tolerant object-based storage system of  claim 9 , wherein obtaining the request to store the N copies of the data object within the fault-tolerant object storage system includes:
 receiving multiple requests to store data within the fault-tolerant object storage system; 
 identifying the N copies of the data object in the data associated with the multiple requests; and 
 generating the request to store the N copies of the data object within the fault-tolerant object storage system. 
 
     
     
       14. A fault-tolerant object-based storage device configured to protect stored data from loss at M data storage entities, each configured to store data on an object-basis, comprising:
 one or more controllers; 
 a non-transitory memory; 
 one or more control links to the M data storage entities; 
 means for obtaining a request to store N copies of a data object within the fault-tolerant object storage system; 
 means for storing the N copies of the data object across the M data storage entities in response to the request, wherein the N copies of the data object are distributed across the M data storage entities such that any two sequential copies of the N copies are stored on two separate storage entities of the M data storage entities; 
 means for generating a first parity object for a first subset of M copies of the N copies of the data object, wherein the first parity object is stored on a first parity storage entity separate from the M data storage entities; 
 means for generating a manifest linking the first parity object with one or more other subsets of M copies of the N copies of the data object 
 means for obtaining a parity compression indicator, wherein the parity compression indicator is user configurable; and 
 means for generating and storing a second parity object for a second subset of M copies of the N copies of the data object when the parity compression indicator is set to off, wherein the second parity object has a same value as the first parity object. 
 
     
     
       15. The fault-tolerant object-based storage device of  claim 14 , wherein the means for obtaining the request to store the N copies of the data object within the fault-tolerant object storage system include:
 means for receiving multiple requests to store data within the fault-tolerant object storage system; 
 means for identifying the N copies of the data object in the data associated with the multiple requests; and 
 means for generating the request to store the N copies of the data object within the fault-tolerant object storage system. 
 
     
     
       16. The fault-tolerant object-based storage device of  claim 14 , further comprising means for restoring a missing data object using a corresponding parity object associated with the missing data object and corresponding data objects stored in the fault-tolerant object storage system.