Apparatus and method for accelerating intrusion detection and prevention systems using pre-filtering

Abstract

An accelerated network intrusion detection and prevention system includes, in part, first, second and third processing stages. The first processing stage receives incoming packets and generates, in response, first and second processed data streams using a first set of rules. The first processing stage optionally detects whether the received packets are suspected of attacking the network and places the received data packets in the first processed data stream. The second processing stage receives the first processed data stream and generates, in response, a third processed data stream using a second set of rules. The second processing stage optionally classifies the first processed data stream, that is suspected of launching a network attack, as either attacks or benign network traffic. A third processing stage receives and processes the second and third processed data streams.

1 . A network intrusion detection system comprising: 
 a first processing stage configured to receive and process one or more input network packets to generate one of at least a first or second processed data streams using a first set of rules;    a second processing stage configured to receive the first processed data stream and to generate in response a third processed data stream using a second set of rules; and    a third processing stage configured to receive and process the second processed data stream from the first processing stage and to receive and process the third processed data stream from the second processing stage.    
   
   
       2 . The system of  claim 1  wherein said first processing stage is further configured to detect one or more suspected network attacks using the received one or more input network packets, wherein said one or more input network packets are included in the transmitted first processed data stream, wherein the first processed data stream is transmitted to the second processing stage for further verification of the one or more suspected network attacks.  
   
   
       3 . The system of  claim 1  wherein said second processing stage is further configured to classify the first processed data stream that is suspected of comprising one or more network attacks as either attacks or benign network traffic.  
   
   
       4 . The system of  claim 1  wherein said second processing stage is further configured to route one or more segments of the first processed data stream to the third processing stage if the first processed data stream is classified as attacks.  
   
   
       5 . The system of  claim 1  wherein said third processing stage is further configured to discard the second and third processed data streams.  
   
   
       6 . The system of  claim 1  wherein said third processing stage comprises one or more second memory segments provided in one or more second memory devices, wherein said first processing stage is further configured to transmit and store the second processed data stream in the one or more second memory segments, wherein said second processing stage is further configured to transmit and store the third processed data stream in the one or more second memory segments.  
   
   
       7 . The system of  claim 1  further comprising: 
 an output module coupled to the first and second processing stages, wherein said first processing stage is further configured to generate a fourth processed data stream, wherein said second processing stage is further configured to generate a fifth processed data stream, wherein said output module is further configured to receive the fourth and fifth processed data streams, the output module being further configured to process the fourth and fifth processed data streams and generate one or more output network packets.    
   
   
       8 . The system of  claim 7  wherein said output module is further configured to derive commands from the fourth and fifth processed data streams, wherein said first processing stage is further configured to derive a first meta data from the input network packets, wherein said first meta data is included in the fourth processed data stream, wherein said second processing stage is further configured to derive a second meta data from the first processed data stream, wherein said second meta data is included in the fifth processed data stream, wherein said commands are included in the output network packets, wherein the commands control the flow of network packets received by the first processing stage.  
   
   
       9 . The system of  claim 1  further comprising: 
 a reporting module coupled to the first and second processing stages, wherein the first processing stage is further configured to generate a sixth processed data stream, wherein said second processing stage is further configured to generate a seventh processed data stream, wherein said reporting module is further configured to receive the sixth and seventh processed data streams, the reporting module being configured to process the sixth and seventh processed data streams, the reporting module being further configured to generate a network security report.    
   
   
       10 . The system of  claim 1  wherein said second processing stage is further configured to derive an eighth processed data stream from the first processed data stream and the second set of rules, the second processing stage being configured to transmit the eighth processed data stream to the first processing stage.  
   
   
       11 . The system of  claim 10  wherein said eighth processed data stream includes a first command and a first command meta data, wherein said first processing stage is configured to classify one or more input network packets as benign packets using the first command and first command meta data included in the eight processed data stream.  
   
   
       12 . The system of  claim 10  wherein said eighth processed data stream includes a second command and a second command meta data, wherein said first processing stage is configured to classify one or more input network packets as attack packets using the second command and second command meta data  
   
   
       13 . The system of  claim 1  wherein said first set of rules is derived from the second set of rules.  
   
   
       14 . The system of  13  wherein said rules include literals and regular expression patterns.  
   
   
       15 . The system of  13  wherein said rules are defined by network and packet characteristics and properties derived from network and packet characteristics.  
   
   
       16 . The system of  claim 1  wherein said first processed data stream includes one or more input network packets.  
   
   
       17 . The system of  claim 1  wherein said first processed data stream includes meta data.  
   
   
       18 . The system of  claim 1  wherein said first processed data stream includes one or more transformed network packets, wherein said first processing stage is further configured to generate one or more transformed network packets from the one or more input network packets.  
   
   
       19 . The system of  claim 9  wherein said second processing stage is further configured to generate classification results, wherein said classification results are included in the seventh processed data stream outputted by the second processing stage, wherein said reporting module is configured to generate a network security report using the classification results derived from the received seventh processed data stream, wherein said network security report comprises alert and logging information  
   
   
       20 . The system of  claim 9  wherein said first processing stage is further configured to generate detection results, wherein said detection results are included in the sixth processed data stream outputted by the first processing stage, wherein said reporting module is configured to generate a network security report using the detection results derived from the received sixth processed data stream, wherein said eighth processed data stream comprises alert and logging information.  
   
   
       21 . The system of  claim 7  wherein said first processing stage is further configured to detect one or more benign input network packets, wherein said one or more benign input network packets are included in the transmitted fourth processed data stream, wherein said fourth processed data stream is transmitted to the output module.  
   
   
       22 . The system of  claim 1  wherein said first processing stage is further configured to identify the one or more input network packets as belonging to one or more streams.  
   
   
       23 . The system of  claim 22  wherein said first processing stage further comprises one or more first memory segments provided in one or more first memory devices coupled to the first processing stage, wherein said first processing stage is further configured to store the one or more input network packets belonging to one or more streams into the one or more first memory segments, wherein the one or more input network packets stored in the one or more first memory segments are included in the first processed data stream generated by the first processing stage.  
   
   
       24 . The system of  claim 7  wherein said first processing stage is further configured to identify the one or more input network packets as belonging to one or more streams.  
   
   
       25 . The system of  claim 24  wherein the one or more input network packets stored in the one or more first memory segments are included in the fourth processed data stream generated by the first processing stage.  
   
   
       26 . The system of  claim 1  wherein said first processing stage is further configured to perform processing on the received one or more input network packets using hardware logic.  
   
   
       27 . The system of  claim 26  wherein said hardware logic is further configured to perform pattern and content processing.  
   
   
       28 . The system of  claim 26  wherein said hardware logic is reconfigurable.  
   
   
       29 . A method for detecting network intrusion, the method comprising: 
 processing one or more input network packets at a first processing stage to generate one of at least a first or second processed data streams using a first set of rules;    generating a third processed data stream at a second processing stage from the first processed data stream and in accordance with a second set of rules; and    supplying the second and third processed data streams to a third processing stage.    
   
   
       30 . The method of  claim 29  further comprising: 
 detecting one or more suspected network attacks using the received one or more input network packets at the first processing stage; and    including in the transmitted first processed data stream the input network packets are included in the transmitted first processed data stream.    
   
   
       31 . The method of  claim 30  wherein said second processing stage is further configured to classify the first processed data stream that is suspected of comprising one or more network attacks as either attacks or benign network traffic.  
   
   
       32 . The method of  claim 31  wherein said second processing stage is further configured to route one or more segments of the first processed data stream to the third processing stage if the first processed data stream is classified as attacks.  
   
   
       33 . The method system of  claim 29  wherein said third processing stage is further configured to discard the second and third processed data streams.  
   
   
       34 . The method of  claim 29  further comprising: 
 storing the second and third processed data streams in a memory.    
   
   
       35 . The method of  claim 29  further comprising: 
 generating a fourth processed data stream;    generating a fifth processed data stream; and    generating one or more output network packets from said fourth and fifth processed data streams.    
   
   
       36 . The method of  claim 29  further comprising: 
 deriving a plurality of commands from the fourth and fifth processed data streams; the commands controlling the flow of network packets received by the first processing stage;    deriving a first meta data from the input network packets;    including the first meta data in the fourth processed data stream;    deriving a second meta data from the first processed data stream;    including the second meta data in the fifth processed data stream; and    including the commands in the output network packets.    
   
   
       37 . The method of  claim 29  further comprising: 
 generating a sixth processed data stream;    generating a seventh processed data stream generating a network security report using said sixth and seventh processed data streams.    
   
   
       38 . The method of  claim 29  further comprising: 
 deriving an eighth processed data stream from the first processed data stream and the second set of rules;    transmitting the eighth processed data stream to the first processing stage.    
   
   
       39 . The method of  claim 38  further comprising: 
 disposing a first command and a first command meta data in said eighth processed data; and    classifying one or more input network packets as benign packets using the first command and first command meta data.    
   
   
       40 . The method of  claim 38  further comprising: 
 disposing a second command and a second command meta data in said eighth processed data; and    classifying one or more input network packets as attack packets using the second command and second command meta data.    
   
   
       41 . The method of  claim 29  wherein the first set of rules is derived from the second set of rules.  
   
   
       42 . The method of  claim 41  wherein said rules include literals and-regular expression patterns.  
   
   
       43 . The method of  claim 41  wherein said rules are defined by network and packet characteristics and properties derived from network and packet characteristics.  
   
   
       44 . The method of  claim 29  wherein said first processed data stream includes one or more input network packets.  
   
   
       45 . The method of  claim 29  wherein said first processed data stream includes meta data.  
   
   
       46 . The method of  claim 29  wherein said first processed data stream includes one or more transformed network packets, wherein said first processing stage is further configured to generate one or more transformed network packets from the one or more input network packets.  
   
   
       47 . The method of  claim 37  wherein said second processing stage is further configured to generate classification results, wherein said classification results are included in the seventh processed data stream outputted by the second processing stage, wherein said reporting module is configured to generate a network security report using the classification results derived from the received seventh processed data stream, wherein said network security report comprises alert and logging information.  
   
   
       48 . The method of  claim 37  wherein said first processing stage is further configured to generate detection results, wherein said detection results are included in the sixth processed data stream outputted by the first processing stage, wherein said reporting module is configured to generate a network security report using the detection results derived from the received sixth processed data stream, wherein said eighth processed data stream comprises alert and logging information.  
   
   
       49 . The method of  claim 35  wherein said first processing stage is further configured to detect one or more benign input network packets, wherein said one or more benign input network packets are included in the transmitted fourth processed data stream, wherein said fourth processed data stream is transmitted to the output module.  
   
   
       50 . The method of  claim 29  wherein said first processing stage is further configured to identify the one or more input network packets as belonging to one or more streams.  
   
   
       51 . The method of  claim 50  wherein said first processing stage further comprises one or more first memory segments provided in one or more first memory devices coupled to the first processing stage, wherein said first processing stage is further configured to store the one or more input network packets belonging to one or more streams into the one or more first memory segments, wherein the one or more input network packets stored in the one or more first memory segments are included in the first processed data stream generated by the first processing stage.  
   
   
       52 . The method of  claim 35  wherein said first processing stage is further configured to identify the one or more input network packets as belonging to one or more streams.  
   
   
       53 . The method of  claim 52  wherein the stored network packets are included in the fourth processed data stream.