Network system to distribute chunks across multiple physical nodes with disk support for object storage

Abstract

A method of storing a file in a storage system that includes a plurality of memory-storage hosts includes: specifying a unique chunk identifier for a memory chunk included in the file; specifying a hash mapping to identify one or more storage locations for the chunk identifier, each storage location corresponding to a portion of a memory-storage host; providing metadata corresponding to the hash mapping to the memory-storage hosts; and storing the memory chuck at the one or more storage locations identified by the hash mapping by providing the chunk identifier to one or more memory-storage hosts corresponding to the identified one or more storage locations, the one or more memory-storage hosts implementing the hash mapping to store the memory chunk at the identified one or more locations.

What is claimed is: 
     
         1 . A method of storing a file in a storage system that includes a plurality of memory-storage hosts, the method comprising:
 specifying a unique chunk identifier for a memory chunk included in the file;   specifying a hash mapping to identify one or more storage locations for the chunk identifier, each storage location corresponding to a portion of a memory-storage host;   providing metadata corresponding to the hash mapping to the memory-storage hosts; and   storing the memory chuck at the one or more storage locations identified by the hash mapping by providing the chunk identifier to one or more memory-storage hosts corresponding to the identified one or more storage locations, the one or more memory-storage hosts implementing the hash mapping to store the memory chunk at the identified one or more locations.   
     
     
         2 . The method of  claim 1 , wherein each memory-storage host maintains a cache memory including values for the metadata and the hash mapping. 
     
     
         3 . The method of  claim 1 , wherein each memory-storage host implements the hash mapping at an interface of said each memory-storage host. 
     
     
         4 . The method of  claim 1 , wherein each memory storage host includes:
 a disk storage unit that is accessible by a addressing a block size and a block offset, and   an interface unit that implements the hash mapping to relate the unique chunk identifier to a corresponding block size and block offset.   
     
     
         5 . The method of  claim 1 , wherein the unique chunk identifier includes a node ID (identification number) to identify a memory-storage host, a LUN ID to identify a Logical Unit Number (LUN) within the memory-storage host, and a Chunk ID to identify a chunk of memory with the LUN. 
     
     
         6 . The method of  claim 1 , wherein the file includes a plurality of memory chunks that have a uniform size given by an integral multiple of a memory block size associated with the storage system. 
     
     
         7 . The method of  claim 1 , further comprising:
 updating the metadata when a memory-storage host is added to the system or deleted from the system; and   providing the updated metadata to the memory-storage hosts.   
     
     
         8 . A non-transitory computer-readable medium that stores a computer program for storing a file in a storage system that includes a plurality of memory-storage hosts, the computer program including instructions that, when executed by at least one computer, cause the at least one computer to perform operations comprising:
 specifying a unique chunk identifier for a memory chunk included in the file;   specifying a hash mapping to identify one or more storage locations for the chunk identifier, each storage location corresponding to a portion of a memory-storage host;   providing metadata corresponding to the hash mapping to the memory-storage hosts; and   storing the memory chuck at the one or more storage locations identified by the hash mapping by providing the chunk identifier to one or more memory-storage hosts corresponding to the identified one or more storage locations, the one or more memory-storage hosts implementing the hash mapping to store the memory chunk at the identified one or more locations.   
     
     
         9 . The non-transitory computer-readable medium of  claim 8 , wherein each memory-storage host maintains a cache memory including values for the metadata and the hash mapping. 
     
     
         10 . The non-transitory computer-readable medium of  claim 8 , wherein each memory-storage host implements the hash mapping at an interface of said each memory-storage host. 
     
     
         11 . The non-transitory computer-readable medium of  claim 8 , wherein each memory storage host includes:
 a disk storage unit that is accessible by a addressing a block size and a block offset, and   an interface unit that implements the hash mapping to relate the unique chunk identifier to a corresponding block size and block offset.   
     
     
         12 . The non-transitory computer-readable medium of  claim 8 , wherein the unique chunk identifier includes a node ID (identification number) to identify a memory-storage host, a LUN ID to identify a Logical Unit Number (LUN) within the memory-storage host, and a Chunk ID to identify a chunk of memory with the LUN. 
     
     
         13 . The non-transitory computer-readable medium of  claim 8 , wherein the file includes a plurality of memory chunks that have a uniform size given by an integral multiple of a memory block size associated with the storage system. 
     
     
         14 . The non-transitory computer-readable medium of  claim 8 , wherein the computer program further includes instructions that, when executed by the at least one computer, cause the at least one computer to perform operations comprising:
 updating the metadata when a memory-storage host is added to the system or deleted from the system; and   providing the updated metadata to the memory-storage hosts.   
     
     
         15 . A system storing a file in a storage system that includes a plurality of memory-storage hosts, the system including at least one computer configured to perform operations for computer-implemented modules including:
 a chunk-identifier module that specifies a unique chunk identifier for a memory chunk included in the file;   a hash-mapping module that specifies a hash mapping to identify one or more storage locations for the chunk identifier, each storage location corresponding to a portion of a memory-storage host;   a storage module that provides metadata corresponding to the hash mapping to the memory-storage hosts and further provides the chunk identifier to one or more memory-storage hosts corresponding to the identified one or more storage locations, the one or more memory-storage hosts implementing the hash mapping to store the memory chunk at the identified one or more locations.   
     
     
         16 . The system of  claim 15 , wherein each memory-storage host maintains a cache memory including values for the metadata and the hash mapping. 
     
     
         17 . The system of  claim 15 , wherein each memory-storage host implements the hash mapping at an interface of said each memory-storage host. 
     
     
         18 . The system of  claim 15 , wherein each memory storage host includes:
 a disk storage unit that is accessible by a addressing a block size and a block offset, and   an interface unit that implements the hash mapping to relate the unique chunk identifier to a corresponding block size and block offset.   
     
     
         19 . The system of  claim 15 , wherein the unique chunk identifier includes a node ID (identification number) to identify a memory-storage host, a LUN ID to identify a Logical Unit Number (LUN) within the memory-storage host, and a Chunk ID to identify a chunk of memory with the LUN. 
     
     
         20 . The system of  claim 15 , wherein the file includes a plurality of memory chunks that have a uniform size given by an integral multiple of a memory block size associated with the storage system.