Method and apparatus for storing and reading files

Abstract

A method and apparatus for storing and reading files are provided. The method includes: dividing a file into a plurality of sections, generating a unique section key for each section, and storing a main key for the file and the plurality of sections keys as a main storage record; dividing a section into a plurality of blocks, generating a unique block value for each block within the section, generating a section value corresponding to the section key based on the plurality of block values, and storing the section key and the section value as a section storage record; and associating each block value with indexing information for the corresponding block. In accordance with the method and apparatus for storing and reading files, file indexes are stored in separate groups to increase the maximum file size and the speed for reading file indexes while reducing the cost for reading file indexes.

1 . A method for storing files, comprising:
 dividing a file into a plurality of sections, generating a unique section key for each section, and storing a main key for the file and the plurality of sections keys as a main storage record;   dividing a section into a plurality of blocks, generating a unique block value for each block within the section, generating a section value corresponding to the section key based on the plurality of block values, and storing the section key and the section value as a section storage record; and   associating each block value with indexing information for the corresponding block.   
     
     
         2 . The method of  claim 1 , further comprising storing the main storage record and the section storage record at different storage devices. 
     
     
         3 . The method of  claim 1 , wherein the section key is stored at an offset to the main key based on an offset of the section within the file. 
     
     
         4 . The method of  claim 1 , wherein each block value is stored in an array at an offset to the section key based on an offset of the block within the section. 
     
     
         5 . The method of  claim 1 , wherein the section key and the corresponding section value are stored in a distributed key-value store. 
     
     
         6 . The method of  claim 1 , wherein the main key and the plurality of section keys are stored in a distributed key-value store. 
     
     
         7 . An apparatus for storing files, comprising:
 a main storage record generation module configured to divide a file into a plurality of sections, generate a unique section key for each section, and store a main key for the file and the plurality of sections keys as a main storage record;   a section storage record generation module configured to dividing a section into a plurality of blocks, generate a unique block value for each block within the section, generate a section value corresponding to the section key based on the plurality of block values, and store the section key and the section value as a section storage record; and   an association module configured to associate each block value with indexing information for the corresponding block.   
     
     
         8 . The apparatus of  claim 7 , wherein the main storage record and the section storage record are stored at different storage devices. 
     
     
         9 . The apparatus of  claim 7 , wherein the section key is stored at an offset to the main key based on an offset of the section within the file. 
     
     
         10 . The apparatus of  claim 7 , wherein each block value is stored in an array at an offset to the section key based on an offset of the block within the section. 
     
     
         11 . The apparatus of  claim 7 , wherein a section key and the corresponding section value are stored in a distributed key-value store. 
     
     
         12 . The apparatus of  claim 7 , wherein the main key and the plurality of section keys are stored in a distributed key-value store. 
     
     
         13 . A method for reading files, comprising:
 determining a main storage record in a key-value store for a file based on a main key;   determining a section storage record and a section value in the key-value store based on a section key; and   determining the location of indexing information of a block based on a block value of the block.   
     
     
         14 . The method of  claim 13 , further comprising reading the main storage record and the section storage record at different storage devices. 
     
     
         15 . The method of  claim 13 , further comprising determining the section key based on an offset of the section within the file. 
     
     
         16 . The method of  claim 13 , further comprising determining the block value based on an offset of the block within the section using a bisection method. 
     
     
         17 . An apparatus for reading files, comprising:
 a main storage record determination module configured to determine a main storage record in a key-value store for a file based on a main key;   a section storage record determination module configured to determine a section storage record and a section value in the key-value store based on a section key; and   a block location determination module configured to determine the location of indexing information of a block based on a block value of the block.   
     
     
         18 . The apparatus of  claim 17 , wherein the main storage record and the section storage record are stored at different storage devices. 
     
     
         19 . The apparatus of  claim 17 , wherein a section key is stored at an offset to the main key based on the offset of the section within the file. 
     
     
         20 . The apparatus of  claim 17 , wherein each block value is stored at an offset to the section key based on an offset of the block within the section.