Data storewidth accelerator

Abstract

Data storage controllers and data storage devices employing lossless or lossy data compression and decompression to provide accelerated data storage and retrieval bandwidth. In one embodiment of the invention, a composite disk controller provides data storage and retrieval acceleration using multiple caches for data pipelining and increased throughput. In another embodiment of the invention, the disk controller with acceleration is embedded in the storage device and utilized for data storage and retrieval acceleration.

1 - 18 . (canceled) 
     
     
         19 . A controller for controlling storage and retrieval of data to and from a data storage device, comprising:
 a data compression/decompression engine for compressing input data for storage to the data storage device and for decompressing output data retrieved from the data storage device, wherein the compressing is performed by a number of encoders in a parallel configuration to produce a plurality of compressed data blocks having a corresponding plurality of associated compression ratios and wherein one of the compressed data blocks is selected and provided as compressed input data based on the compression ratio associated with the selected compressed data block;   a first cache, operatively connected to the storage device and the data compression/decompression engine, for temporary storage of (i) compressed output data obtained by reading from the data storage device and (ii) the compressed input data obtained from the data compression/decompression engine for writing to the data storage device, wherein the data compression/decompression engine operates such that at least one of the following conditions is satisfied:   (a) the combined length of time required for performing said compressing input data and said writing to the data storage device is less than a length of time required for writing input data to the data storage device; and   (b) the combined length of time required for performing said reading from the data storage device and said decompressing output data is less than a length of time required for reading output data from the data storage device;   a second cache, operatively connected to the data compression/decompression engine and to a host interface bus, for temporary storage of (i) data that is received from a host system over the host interface bus for compression and storage and (ii) data that is to be sent to the host system over the host interface bus; and   a cache manager for controlling the first and second data caches under commands received from the data compression/decompression engine.   
     
     
         20 . The controller of  claim 19 , wherein the controller comprises a disk controller and wherein the storage device comprises a hard disk. 
     
     
         21 . The controller of  claim 19 , wherein the first cache, second cache and the data compression/decompression engine are operatively connected by a first local bus. 
     
     
         22 . The controller of  claim 19 , wherein the data compression/decompression engine is connected to the cache manager over a dedicated bus. 
     
     
         23 . The controller of  claim 19 , further comprising a virtual file system. 
     
     
         24 . The controller of  claim 19 , wherein the controller comprises a magnetic disk controller. 
     
     
         25 . The controller of  claim 19 , wherein the data compression/decompression engine, the first cache, and the second cache are implemented on a digital signal processor. 
     
     
         26 . The controller of  claim 19 , wherein the first cache, second cache and data compression/decompression engine are embedded components of the controller. 
     
     
         27 . A system comprising:
 an embedded data compression/decompression engine for compressing input data for storage to a storage device and for decompressing output data retrieved from the storage device, wherein the compressing is performed by a number of encoders in a parallel configuration to produce a plurality of compressed data blocks having a corresponding plurality of associated compression ratios and wherein one of the compressed data blocks is selected and provided as compressed input data based on the compression ratio associated with the selected compressed data block;   an embedded first cache, operatively connected to the storage device and the data compression/decompression engine, for temporary storage of (i) compressed output data obtained by reading from the storage device and (ii) the compressed input data obtained from the data compression/decompression engine for writing to the storage device, wherein the embedded data compression/decompression engine operates such that at least one of the following conditions is satisfied:   (a) the combined length of time required for performing said compressing input data and said writing to the storage device is less than a length of time required for writing input data to the storage device; and   (b) the combined length of time required for performing said reading from the storage device and said decompressing output data is less than a length of time required for reading output data from the storage device;   an embedded second cache, operatively connected to the data compression/decompression engine and to a host interface bus, for temporary storage of (i) data that is received from a host system over the host interface bus for compression and storage and (ii) data that is to be sent to the host system over the host interface bus; and   an embedded cache manager for controlling the first and second data caches under commands received from the data compression/decompression engine.   
     
     
         28 . The system of  claim 27 , wherein the first cache, second cache and the data compression/decompression engine are operatively connected by a first local bus. 
     
     
         29 . The system of  claim 27 , wherein the data compression/decompression engine is connected to the cache manager over a dedicated bus. 
     
     
         30 . The system of  claim 27 , further comprising an embedded virtual file system. 
     
     
         31 . The system of  claim 27 , wherein the data compression/decompression engine, the first cache, and the second cache are implemented on a digital signal processor. 
     
     
         32 . A storage device controller for controlling storage and retrieval of data to and from a storage device, comprising:
 an embedded data compression/decompression engine for compressing input data for storage to the storage device and for decompressing output data retrieved from the storage device, wherein the compressing is performed by a number of encoders in a parallel configuration to produce a plurality of compressed data blocks having a corresponding plurality of associated compression ratios and wherein one of the compressed data blocks is selected and provided as compressed input data based on the compression ratio associated with the selected compressed data block;   an embedded bi-directional cache for temporary storage of (i) compressed output data obtained by reading from the storage device and (ii) the compressed input data obtained from the data compression/decompression engine for writing to the storage device, wherein the embedded data compression/decompression engine operates such that at least one of the following conditions is satisfied:   (a) the combined length of time required for performing said compressing input data and said writing to the storage device is less than a length of time required for writing input data to the storage device; and   (b) the combined length of time required for performing said reading from the storage device and said decompressing output data is less than a length of time required for reading output data from the storage device;   an embedded cache manager for controlling the bi-directional cache; and   an embedded virtual file management system for mapping compressed data blocks stored on the storage device to corresponding uncompressed data blocks.   
     
     
         33 . The system of  claim 27 , wherein said storage device is a magnetic disk. 
     
     
         34 . The controller of  claim 32 , wherein the storage device is a magnetic disk. 
     
     
         35 . A system comprising:
 a data compression/decompression engine for compressing input data for storage to a data storage device and for decompressing output data retrieved from the data storage device, wherein the compressing is performed by a number of encoders in a parallel configuration to produce a plurality of compressed data blocks having a corresponding plurality of associated compression ratios and wherein one of the compressed data blocks is selected and provided as compressed input data based on the compression ratio associated with the selected compressed data block;   a first cache, operatively connected to the storage device and the data compression/decompression engine, for temporary storage of (i) compressed output data obtained by reading from the data storage device and (ii) the compressed input data obtained from the data compression/decompression engine for writing to the data storage device, wherein the data compression/decompression engine operates such that   (a) the combined length of time required for performing said compressing input data and said writing to the data storage device is less than a length of time required for writing input data to the data storage device; and   (b) the combined length of time required for performing said reading from the data storage device and said decompressing output data is less than a length of time required for reading output data from the data storage device;   a second cache, operatively connected to the data compression/decompression engine and to a host interface bus, for temporary storage of (i) data that is received from a host system over the host interface bus for compression and storage and (ii) data that is to be sent to the host system over the host interface bus; and   a cache manager for controlling the first and second data caches.   
     
     
         36 . The system of  claim 35 , wherein the storage device is a magnetic disk.