Intelligent data storage and processing using fpga devices

Abstract

A data storage and retrieval device and method is disclosed. The device includes at least one magnetic storage medium configured to store target data and at least one re-configurable logic device comprising an FPGA coupled to the at least one magnetic storage medium and configured to read a continuous stream of target data therefrom, having been configured with a template or as otherwise desired to fit the type of search and data being searched. The re-configurable logic device is configured to receive at least one search inquiry in the form of a data key and to determine a match between the data key and the target data as it is being read from the at least one magnetic storage medium. This device and method can perform a variety of searches on the target data including without limitation exact and approximate match searches, sequence match searches, image match searches and data reduction searches. This device and method may be provided as part of a stand-alone computer system, embodied in a network attached storage device, or can otherwise be provided as part of a computer LAN or WAN. In addition to performing search and data reduction operations, this device may also be used to perform a variety of other processing operations including encryption, decryption, compression, decompression, and combinations thereof.

1 . A method for securely sharing data with authorized parties, wherein the data to be shared is stored in a database in a first encrypted format, the method comprising: 
 providing a programmable logic device for connection to the database, wherein the programmable logic device is configured to (1) receive a stream of encrypted data from the database, (2) decrypt the received encrypted data stream to create decrypted data, and (3) encrypt the decrypted data in a second encrypted format; and    sharing the data of the second encrypted format by communicating it to an authorized party.    
     
     
         2 . The method of  claim 1  further comprising: 
 providing the authorized party with a key to decrypt the shared data.    
     
     
         3 . The method of  claim 1  wherein the second encrypted format is different than the first encrypted format such that the key provided to the authorized party will be different than a key necessary to decrypt the stored data.  
     
     
         4 . The method of  claim 1  further comprising providing a memory device in communication with the programmable logic device, wherein the content of the memory device is accessible only by the programmable logic device, and wherein the programmable logic device is further configured to store at least a portion of the decrypted data in the memory device.  
     
     
         5 . The method of  claim 1  wherein the database is owned by a first party, and wherein the data stored in the database is owned by a second party.  
     
     
         6 . The method of  claim 2  further comprising: 
 receiving a request for stored data from the authorized party;    responsive to the received request, retrieving stored data from the database; and    processing the stored data through the programmable logic device.    
     
     
         7 . The method of  claim 1  further comprising storing data in the database in the first encrypted format.  
     
     
         8 . The method of  claim 1  wherein the programmable logic device is an FPGA.  
     
     
         9 . A device for preparing stored encrypted data for communication to a party, the device comprising: 
 a programmable logic device in communication with a data storage medium, the data storage medium comprising data stored therein in a first encrypted format, the programmable logic device being configured to (1) decrypt a stream of encrypted data received from the data storage medium to thereby create decrypted data, and (2) encrypt the decrypted data in a second encrypted format.    
     
     
         10 . The device of  claim 9  wherein the second encrypted format is different than the first encrypted format such that a key provided to the party to decrypt the data of the second encrypted format will be different than a key necessary to decrypt the stored data.  
     
     
         11 . The device of  claim 9 , wherein the device is in communication with a processor, the processor being configured to send a request to the device for stored data, the request to be fulfilled at least in part by the programmable logic device, the device further comprising a memory device in communication with the programmable logic device, wherein the content of the memory device is accessible by the programmable logic device but is not accessible by the processor, and wherein the programmable logic device is further configured to store at least a portion of the decrypted data in the memory device.  
     
     
         12 . The device of  claim 9  wherein the programmable logic device is also configured to perform a socket operation on incoming and outgoing data to interface the programmable logic device with upstream and downstream components.  
     
     
         13 . The device of  claim 9  wherein the data storage medium comprises a hard disk drive system, the device further comprising a disk connector for interfacing the device with the hard disk drive system.  
     
     
         14 . The device of  claim 13  further comprising a disk controller in communication with the disk connector and the programmable logic device.  
     
     
         15 . The device of  claim 14  further comprising an internal bus connecting the disk controller with the programmable logic device.  
     
     
         16 . The device of  claim 15  wherein the internal bus is a PCI-X bus.  
     
     
         17 . The device of  claim 16  further comprising a bus connector for interfacing the programmable logic device with a bus on a computer motherboard.  
     
     
         18 . The device of  claim 17  wherein the bus connector is a PCI-X bus connector.  
     
     
         19 . The device of  claim 12  wherein the programmable logic device is an FPGA.  
     
     
         20 . A method for securely sharing data, wherein the data to be shared is stored in a database in a first encrypted format, the method comprising: 
 providing a reconfigurable logic device for connection to the database, wherein the programmable logic device is configured to (1) receive a stream of encrypted data from the database, and (2) using reconfigurable hardware logic, translate the received stream from the first encrypted format to a second encrypted format different than the first encrypted format; and    delivering the data of the second encrypted format to a requester.    
     
     
         21 . The method of  claim 20  wherein the requester is an authorized requester, the method further comprising: 
 providing the authorized requester with a means for decrypting the delivered data.    
     
     
         22 . The method of  claim 20  wherein the database is owned by a first party, and wherein the data stored in the database is owned by a second party.  
     
     
         23 . The method of  claim 20  wherein the reconfigurable logic device is an FPGA, the method further comprising: 
 receiving a request for stored data from the requester;    responsive to the received request, retrieving stored data from the database; and    processing the stored data through the programmable logic device.    
     
     
         24 . A device for processing data received from a mass storage medium, the device comprising: 
 a programmable logic device in communication with the mass storage medium, the mass storage medium comprising compressed data stored therein, the programmable logic device being configured to (1) receive a continuous stream of compressed data from the mass storage medium, (2) decompress the received compressed data stream to create a stream of decompressed data, and (3) perform a search operation within the stream of decompressed data.    
     
     
         25 . The device of  claim 24  wherein the search operation is configured to determine whether a pattern match exists between a search key that is representative of data desired to be retrieved from the mass storage medium and a data signal that is representative of a decompressed data stream.  
     
     
         26 . The device of  claim 24  further comprising a memory device in communication with the programmable logic device, wherein the content of the memory device is accessible only by the programmable logic device, and wherein the programmable logic device is further configured to store at least a portion of the decrypted data in the memory device.  
     
     
         27 . The device of  claim 24  wherein the programmable logic device is also configured to perform a socket operation on incoming and outgoing data to interface the programmable logic device with upstream and downstream components.  
     
     
         28 . The device of  claim 24  wherein the storage medium comprises a hard disk drive system, the device further comprising a disk connector for interfacing the device with the hard disk drive system.  
     
     
         29 . The device of  claim 28  further comprising a disk controller in communication with the disk connector and the programmable logic device.  
     
     
         30 . The device of  claim 29  further comprising an internal bus connecting the disk controller with the programmable logic device.  
     
     
         31 . The device of  claim 30  wherein the internal bus is a PCI-X bus.  
     
     
         32 . The device of  claim 31  further comprising a bus connector for interfacing the programmable logic device with a bus on a computer motherboard.  
     
     
         33 . The device of  claim 32  wherein the bus connector is a PCI-X bus connector.  
     
     
         34 . The device of  claim 27  wherein the programmable logic device comprises an FPGA.  
     
     
         35 . A device for processing data received from a mass storage medium, the device comprising: 
 a reconfigurable logic device in communication with the mass storage medium, the mass storage medium comprising compressed data stored therein, the reconfigurable logic device being configured to perform a plurality of processing operations with reconfigurable hardware logic on a continuous stream of compressed data it receives from the mass storage medium, wherein the plurality of processing operations comprise at least a decompression operation followed by a search operation.    
     
     
         36 . The device of  claim 35  wherein the reconfigurable logic device is an FPGA, and wherein the search operation is configured to determine whether a pattern match exists between a search key that is representative of data desired to be retrieved from the mass storage medium and a data signal that is representative of a decompressed data stream exiting the decompression operation.  
     
     
         37 . The device of  claim 35  wherein the reconfigurable logic device is a programmable logic device, the device further comprising a memory device in communication with the programmable logic device, wherein the content of the memory device is accessible by the programmable logic device but is not accessible by a remote processor in communication with the device, and wherein the programmable logic device is further configured to store at least a portion of a decompressed data output from the decompression operation in the memory device.  
     
     
         38 . A method searching data stored in a storage medium in a compressed format, the method comprising: 
 providing a programmable logic device for connection to the storage medium, wherein the programmable logic device is configured to (1) receive a stream of compressed data from the storage medium, (2) decompress the received compressed data stream to create decompressed data, and (3) perform a search operation on the decompressed data.    
     
     
         39 . The method of  claim 38  further comprising receiving a search query from a remote processor, and wherein the programmable logic device is configured to perform the search operation in accordance with the query, and wherein the method further comprises delivering a search result from the search operation to the remote processor.  
     
     
         40 . The method of  claim 39  wherein the search operation is configured to determine whether a pattern match exists between a search key that is representative of data desired to be retrieved from the mass storage medium and a data signal that is representative of a decompressed data stream.  
     
     
         41 . The method of  claim 38  further comprising providing a memory device in communication with the programmable logic device, wherein the content of the memory device is accessible only by the programmable logic device, and wherein the programmable logic device is further configured to store at least a portion of the decompressed data in the memory device.  
     
     
         42 . A system for processing data received from a mass storage medium, the device comprising: 
 a storage medium in which at least some data is stored in a compressed format; and    a programmable logic device in communication with the storage medium, the programmable logic device being configured to (1) receive a continuous stream of compressed data from the storage medium, (2) decompress the received compressed data stream to create a stream of decompressed data, and (3) perform a search operation within the stream of decompressed data.    
     
     
         43 . The system of  claim 42  wherein the search operation is configured to determine whether a pattern match exists between a search key that is representative of data desired to be retrieved from the mass storage medium and a data signal that is representative of a decompressed data stream.  
     
     
         44 . The system of  claim 42  further comprising a memory device in communication with the programmable logic device, wherein the content of the memory device is accessible only by the programmable logic device, and wherein the programmable logic device is further configured to store at least a portion of the decompressed data in the memory device.