Relational objects for the optimized management of fixed-content storage systems

Abstract

A system and method is described for managing data objects in a fixed-content storage system. In one embodiment, differenced objects are created when an object stored in a fixed-content storage system is edited. The edits to the original object may represent a small change in the original object, but because the stored original object is immutable it is not possible to simply overwrite the small portion that is edited. In order to store the edited data without requiring duplication of existing data, a new object is created that references both the original object and the edited data. The metadata of the new object includes information relating to the offset and the size of the edited data so that the edited data is accessed instead of the corresponding portion of the original object.

1 . A method of reducing duplicative storage of data on a fixed-content storage system, the method comprising:
 maintaining, on a fixed-content storage system comprising at least one computer-readable storage device, a first data object;   receiving a second data object to be stored on the fixed-content storage system;   identifying, using at least one computer processor, a first portion of the second data object and a second portion of the second data object, wherein the first portion of the second data object comprises data identical to a portion of the first data object;   constructing a differenced object using the at least one computer processor, wherein the differenced object comprises the second portion of the second data object, wherein the differenced object further comprises a reference to the portion of the first data object that is identical to the first portion of the second data object; and   storing the differenced object on the fixed-content storage system.   
     
     
         2 . The method of  claim 1 , wherein the differenced object further comprises metadata configured to enable reconstruction of the contents of the second data object, and whereby the differenced object does not include the first portion of the second object. 
     
     
         3 . The method of  claim 1 , further comprising calculating a storage size associated with differences between the first object and the second object, wherein the differenced object is stored on the fixed-content storage system based at least on a determination that the calculated storage size satisfies a specified requirement. 
     
     
         4 . The method of  claim 1 , wherein the fixed-content storage system comprises a plurality of distributed nodes, each distributed node comprising at least one processor and at least one storage device, and wherein the one or more computer processors are configured to store the consolidated data object on more than one distributed node. 
     
     
         5 . The method of  claim 4 , wherein the plurality of distributed nodes spans multiple geographically separated sites, and wherein at least a portion of the plurality of distributed nodes are configured to communicate on a network. 
     
     
         6 . The method of  claim 1 , wherein the differenced object is associated with one or more rules relating to data retention and replication, the fixed-content storage system being configured to retain and replicate the differenced object in accordance with the one or more rules. 
     
     
         7 . The method of  claim 6 , wherein the determination that the calculated storage size satisfies a specified requirement comprises a determination that the calculated storage size meets a threshold level in relation to a size associated with the first object. 
     
     
         8 . The method of  claim 1 , further comprising:
 determining that the first object may be removed from the fixed-content storage system;   constructing a flattened object based on the differenced object by combining the second portion of the second object and the portion of the first object that is identical to the first portion of the second object, wherein the flattened object does not include a reference to the first object;   storing the flattened object on the fixed-content storage system; and   removing the first object from the fixed-content storage system.   
     
     
         9 . A computing system configured to store data objects, the computing system comprising:
 a fixed-content storage system comprising one or more computer-readable storage devices; and   one or more computer processors in communication with the fixed-content storage system;   the fixed-content storage system configured to maintain a first data object;   the one or more computer processors configured to receive a second data object to be stored on the fixed-content storage system;   the one or more processors configured to identify a first portion of the second data object and a second portion of the second data object, wherein the first portion of the second data object comprises data identical to a portion of the first data object;   the one or more processors configured to construct a differenced object using the at least one computer processor, wherein the differenced object comprises the second portion of the second data object, wherein the differenced object further comprises a reference to the portion of the first data object that is identical to the first portion of the second data object; and   the one or more processors configured to store the differenced object on the fixed-content storage system.   
     
     
         10 . The computing system of  claim 9 , wherein the differenced object further comprises metadata configured to enable reconstruction of the contents of the second data object, whereby the differenced object does not include the first portion of the second object. 
     
     
         11 . The computing system of  claim 9 , wherein the one or more processors configured to calculate a storage size associated with differences between the first object and the second object, wherein the one or more processors are configured to store the differenced object on the fixed-content storage system based at least on a determination that the calculated storage size satisfies a specified requirement. 
     
     
         12 . The computing system of  claim 11 , wherein the determination that the calculated storage size satisfies a specified requirement comprises a determination that the calculated storage size meets a threshold level in relation to a size associated with the first object. 
     
     
         13 . The computing system of  claim 9 , wherein the one or more processors are further configured to:
 determine that the first object may be removed from the fixed-content storage system;   construct a flattened object based on the differenced object by combining the second portion of the second object and the portion of the first object that is identical to the first portion of the second object, wherein the flattened object does not include a reference to the first object;   store the flattened object on the fixed-content storage system; and   remove the first object from the fixed-content storage system.   
     
     
         14 . The computing system of  claim 9 , wherein the fixed-content storage system comprises a plurality of distributed nodes, each distributed node comprising at least one processor and at least one storage device, and wherein the one or more computer processors are configured to store the consolidated data object on more than one distributed node. 
     
     
         15 . The computing system of  claim 14 , wherein the plurality of distributed nodes spans multiple geographically separated sites, and wherein at least a portion of the plurality of distributed nodes are configured to communicate on a network. 
     
     
         16 . The computing system of  claim 9 , wherein the differenced object is associated with one or more rules relating to data retention and replication, the fixed-content storage system being configured to retain and replicate the differenced object in accordance with the one or more rules. 
     
     
         17 . A non-transitory computer-readable medium having stored thereon a plurality of executable instructions configured to be executed on a fixed-content storage system having stored thereon a first data object, the executable instructions configured to cause the fixed-content storage system to perform operations comprising:
 receiving a second data object;   identifying a first portion of the second data object and a second portion of the second data object, wherein the first portion of the second data object corresponds to a portion of the first data object;   constructing a third object using the at least one computer processor, wherein the third object comprises the second portion of the second data object, wherein the third object further comprises a reference to the portion of the first data object that corresponds to the first portion of the second data object; and   storing the third object on the fixed-content storage system.   
     
     
         18 . The non-transitory computer-readable medium of  claim 17 , wherein the third object is further configured to enable reconstruction of the contents of the second data object. 
     
     
         19 . The non-transitory computer-readable medium of  claim 17 , wherein the executable instructions are further configured to be executed on a plurality of distributed nodes, each distributed node comprising at least one processor and at least one storage device, and wherein the one or more computer processors are configured to store the consolidated data object on more than one distributed node, wherein the plurality of distributed nodes spans multiple geographically separated sites, and wherein at least a portion of the plurality of distributed nodes are configured to communicate on a network. 
     
     
         20 . The non-transitory computer-readable medium of  claim 17 , wherein the third object is associated with one or more rules relating to data retention and replication, the fixed-content storage system being configured to retain and replicate the third data object in accordance with the one or more rules.