Real-time adaptive processing of network data packets for analysis

Abstract

A network monitoring system that summarizes a plurality of data packets of a session into a compact session record for storage and processing. Each session record may be produced in real-time and made available during the session and/or after the termination of the session. Depending on protocols, a network monitoring system extracts different sets of information, removes redundant information from the plurality of data packets, and adds performance information to produce the session record. The network monitoring system may retrieve and process a single session record or multiple session records for the same or different protocols to determine cause of events, resolve issues in a network or evaluate network performance or conditions. The session record enables analysis in the units of session instead of individual packets. Hence, the network monitoring system can analyze events, issues or performance of the network more efficiently and effectively.

What is claimed is: 
     
       1. A method, comprising:
 determining, by a processor executing computer-readable instructions stored on a memory, first and second sessions associated with data packets received by a network monitoring device at a location in from a network; 
 generating, by the processor, a common header for each of the first and second sessions by summarizing information in headers of a plurality of data packets in associated with each of the first and second sessionsby the network monitoring device; and 
 generating, by the processor, for each of the first and second session sessions, a session record for respectively storing comprising the common header of the first and second session sessions, respectively, and storing mapping information representing association of the first session with the second session when the first and second sessions are associated with planes in a same protocol, 
 wherein the session record of each of the first and second sessions further stores links to the plurality of data packets respectively in each of the first and second sessions. 
 
     
     
       2. The method as recited in  claim 1 , further including comprising: 
 generating, by the processor, a unit of payload metadata summarizing information in payloads of the plurality of the data packets in each of the first and second session by the network monitoring device sessions; and  
 wherein the generating step further includes storing, by the processor, the payload metadata of each of the first and second session respectively sessions in the session record for each the first and second session sessions, respectively. 
 
     
     
       3. The method of  claim 2 , further comprising determininga, by the processor, the protocol associated with each of the first and second session sessions, wherein the information included in at least one of the common header or the payload metadata of each the first and second session sessions is respectively determined based on a the protocol associated with the first and second session sessions. 
     
     
       4. The method of  claim 2 , wherein the common header and the payload metadata are generated in real-time responsive to receiving the data packets for each of the first and second session sessions from the network. 
     
     
       5. The method of  claim 1 , further comprising:
 generating, by the processor, performance parameters representing performance of at least one of the network, an application associated with the network, a service associated with the network, a client associated with the network, or a server associated with the network for transmitting the plurality of the data packets in each of the first and second sessions; and 
 adding, by the processor, the performance parameters of the first and second sessions respectively in the first and second session record of the first and second sessions, respectively. 
 
     
     
       6. The method of  claim 5 , wherein the performance parameters comprise at least one of a packet loss rate, a response time, a number of packets or bytes transmitted, parameters indicative of transmission latency, codec parameters, total transfer time for the session, R-Factor (rating factor), MOS (Mean Opinion Score), a file or page size, a file name, sender/receive information, a total transfer time, a user ID, jitter, and phone number. 
     
     
       7. The method of  claim 1 , wherein the first session is associated with a first protocol and the second session is associated with a second protocol different than the first protocol. 
     
     
       8. The method of  claim 1 , wherein each first and second session record further stores links to the plurality of the data packets respectively in each of the first and second sessions. 
     
     
       9. The method of  claim 1 , wherein the common header comprises fields of the headers that remain unchanged throughout each respective of the first and second session sessions. 
     
     
       10. A network monitoring device for processing network data for analysis, comprising:
 a network interface configured to receive data packets at a location in a network; and 
 a session tracing engine configured processor executing computer-readable instructions stored on a memory to:
 determine first and second sessions associated with data packets received by a network monitoring device at a location in from a network; 
 generate a common header for each of the first and second sessions by summarizing information in headers of a plurality of data packets in associated with each of the first and second sessionsby the network monitoring device; and 
 generate, for each of the first and second session sessions, a session record for respectively storing comprising the common header of the first and second session sessions, and storing mapping information representing association of the first session with the second session when the first and second sessions are associated with planes in a same protocol. 
 
 
     
     
       11. The network monitoring device as recited in of  claim 10 , wherein the session tracing engine is processor further configured executes the computer-readable instructions to: 
 generate a unit of payload metadata summarizing information in payloads of the plurality of the data packets in each of the first and second session by the network monitoring device sessions; and  
 wherein the generating step further includes storingstore the payload metadata of each of the first and second session respectivelysessions in the session record for each of the first and second sessionsessions. 
 
     
     
       12. The network monitoring device as recited in of  claim 11 , wherein the session tracing engine is further configured to processor further executes the computer-readable instructions to determine a protocol associated with each of the first and second session sessions, and wherein the information included in at least one of the common header or the payload metadata of each first and second session sessions is respectively determined based on a the protocol associated with the first and second session sessions. 
     
     
       13. The network monitoring device as recited in of  claim 11 , wherein the common header and the payload metadata are generated in real-time responsive to receiving the data packets for each of the first and second session sessions from the network. 
     
     
       14. The network monitoring device as recited in of claim  11  10, wherein each first and second the session record of each of the first and second sessions further stores links to the plurality of the data packets respectively in each of the first and second sessions. 
     
     
       15. The network monitoring device as recited in of  claim 10 , wherein the session tracing engine is processor further configured executes the computer-readable instructions to:
 generate performance parameters representing performance of at least one of the network, an application associated with the network, a service associated with the network, a client associated with the network, or a server associated with the network for transmitting the plurality of the data packets in each of the first and second sessions; and 
 add the performance parameters of the first and second sessions respectively in the first and second session record of the first and second sessions. 
 
     
     
       16. The network monitoring device as recited in of  claim 15 , wherein the performance parameters comprise at least one of a packet loss rate, a response time, a number of packets or bytes transmitted, parameters indicative of transmission latency, codec parameters, total transfer time for the session, R-Factor (rating factor), MOS (Mean Opinion Score), a file or page size, a file name, sender/receive information, a total transfer time, a user ID, jitter, and phone number. 
     
     
       17. The network monitoring device as recited in of  claim 10 , wherein the first session is associated with a first protocol and the second session is associated with a second protocol different than the first protocol. 
     
     
       18. The network monitoring device as recited in of  claim 10 , wherein the common header comprises fields of the headers that remain unchanged throughout each respective of the first and second session sessions. 
     
     
       19. A non-transitory computer-readable media having computer-readable instructions stored thereon that when executed by a processor cause the processor to perform a process comprising:
 determining first and second sessions associated with packets received from a network;   generating a common header for each of the first and second sessions by summarizing information of a plurality of packets associated with each of the first and second sessions; and   generating for each of the first and second sessions, a session record comprising the common header of the first and second sessions, respectively, and storing mapping information representing association of the first session with the second session when the first and second sessions are associated with a protocol,   wherein the session record of each of the first and second sessions further stores links to the plurality of data packets respectively in each of the first and second sessions.   
     
     
       20. The non-transitory computer-readable media of claim 19, wherein the processor further executes the computer-readable instructions to:
 generate a unit of payload metadata summarizing information in payloads of the plurality of packets in each of the first and second sessions; and   store the payload metadata of each of the first and second sessions respectively in the session record for each of the first and second sessions.   
     
     
       21. The non-transitory computer-readable media of claim 20, wherein the processor further executes the computer-readable instructions to determine a protocol associated with each of the first and second sessions, wherein the information included in at least one of the common header or the payload metadata of each first and second sessions is respectively determined based on the protocol associated with the first and second sessions. 
     
     
       22. The non-transitory computer-readable media of claim 20, wherein the common header and the payload metadata are generated in real-time responsive to receiving the packets for each of the first and second sessions from the network. 
     
     
       23. The non-transitory computer-readable media of claim 19, wherein the processor further executes the computer-readable instructions to:
 generate performance parameters representing performance of at least one of the network, an application associated with the network, a service associated with the network, a client associated with the network, or a server associated with the network for transmitting the plurality of packets in each of the first and second sessions; and   add the performance parameters of the first and second sessions respectively in the session record of the first and second sessions.   
     
     
       24. The non-transitory computer-readable media of claim 19, wherein the first session is associated with a first protocol and the second session is associated with a second protocol different than the first protocol. 
     
     
       25. A non-transitory computer-readable media comprising computer-readable instructions stored thereon that when executed by a processor cause the processor to perform a process comprising:
 determining a session associated with a packet received from a network;   generating a common header by summarizing first information of the packet and at least one other packet of a same session;   generating a payload metadata by summarizing second information of the packet and the at least one other packet by:
 determining a data payload from the packet and each of the at least one other packet; and 
 storing a subset of the data payload as the payload metadata, wherein the subset of the data payload is based on a protocol associated with the session; and 
   generating a session record comprising the common header and the payload metadata for evaluating at least one of the network, an application associated with the network, or a service associated with the network.   
     
     
       26. The non-transitory computer-readable media of claim 25, wherein the processor further executes the computer-readable instructions to buffer data associated with the packet and the at least one other packet before generating the common header and the payload metadata. 
     
     
       27. The non-transitory computer-readable media of claim 25, wherein the processor further executes the computer-readable instructions to determine the protocol associated with the session based upon information associated with the packet and the at least one other packet. 
     
     
       28. The non-transitory computer-readable media of claim 25, wherein the processor further executes the computer-readable instructions to:
 assign a new session identifier (ID) to the session upon determining that the session is a new session or assign an existing session ID to the session upon determining that the session is a pre-existing session, wherein the existing session ID is associated with the pre-existing session; and   generate session information based on the new session ID or the existing session ID and the protocol associated with the session.   
     
     
       29. The non-transitory computer-readable media of claim 25, wherein for generating the common header by summarizing the first information, the processor executes the computer-readable instructions to:
 extract static data from a header of the packet and the header of each of the at least one other packet; and   store the static data in the common header.   
     
     
       30. The non-transitory computer-readable media of claim 25, wherein for generating the common header by summarizing the first information, the processor executes the computer-readable instructions to summarize non-static data that is changed in the packet and the at least one other packet. 
     
     
       31. The non-transitory computer-readable media of claim 25, wherein for generating the common header by summarizing the first information, the processor executes the computer-readable instructions to extract the first information from the packet and each of the at least one other packet, and wherein the first information is based upon the protocol associated with the session. 
     
     
       32. The non-transitory computer-readable media of claim 25, wherein the common header and the payload metadata are generated in real-time responsive to receiving the packet and the at least one other packet from the network. 
     
     
       33. The non-transitory computer-readable media of claim 25, wherein the processor further executes the computer-readable instructions to:
 generate a performance parameter representing performance at least one of the network, the application, the service, a client associated with the network, or a server associated with the network, wherein the performance parameter is generated based on the packet and the at least one other packet; and   include the performance parameter in the session record.   
     
     
       34. The non-transitory computer-readable media of claim 25, wherein the processor further executes the computer-readable instructions to generate mapping information based on the session and at least one other session, and wherein the mapping information is included in the session record. 
     
     
       35. The non-transitory computer-readable media of claim 34, wherein the session and the at least one other session are associated with a same protocol. 
     
     
       36. The non-transitory computer-readable media of claim 34, wherein the session is associated with a first protocol and the at least one other session is associated with a second protocol different than the first protocol. 
     
     
       37. The non-transitory computer-readable media of claim 25, wherein the session and at least one other session are associated with a same protocol. 
     
     
       38. The non-transitory computer-readable media of claim 25, wherein the session is associated with a first protocol and at least one other session is associated with a second protocol different than the first protocol. 
     
     
       39. A network monitoring device comprising:
 a memory storing computer-readable instructions; and   a processor that executes the computer-readable instructions to:
 determine a session associated with a packet received from a network; 
 generate a common header by summarizing first information of the packet and at least one other packet of a same session; 
 generate a payload metadata by summarizing second information of the packet and the at least one other packet; and 
 generate a session record comprising the common header and the payload metadata to evaluate at least one of the network, an application associated with the network, or a service associated with the network. 
   
     
     
       40. The network monitoring device of claim 39, wherein the processor further executes the computer-readable instructions to determine a protocol associated with the session based upon information associated with the packet and the at least one other packet. 
     
     
       41. The network monitoring device of claim 39, wherein the processor further executes the computer-readable instructions to generate the common header and the payload metadata in real-time responsive to receiving the packet and the at least one other packet from the network. 
     
     
       42. The network monitoring device of claim 39, wherein the processor further executes the computer-readable instructions to:
 generate a performance parameter representing performance at least one of the network, the application, the service, a client associated with the network, or a server associated with the network, wherein the performance parameter is generated based on the packet and the at least one other packet; and   include the performance parameter in the session record.   
     
     
       43. The network monitoring device of claim 39, wherein the processor further executes the computer-readable instructions to generate mapping information based on the session and at least one other session, and wherein the mapping information is included in the session record. 
     
     
       44. The network monitoring device of claim 43, wherein the session and the other session are associated with a same protocol. 
     
     
       45. The network monitoring device of claim 43, wherein the session is associated with a first protocol and the at least one other session is associated with a second protocol different than the first protocol. 
     
     
       46. The network monitoring device of claim 39, wherein the processor further executes the computer-readable instructions to buffer data associated with the packet and the at least one other packet before generating the common header and the payload metadata. 
     
     
       47. The network monitoring device of claim 39, wherein the processor further executes the computer-readable instructions to:
 assign a new session ID to the session upon determining that the session is a new session or assign an existing session ID to the session upon determining that the session is a pre-existing session, wherein the existing session ID is associated with the pre-existing session; and   generate session information based on the new session ID or the existing session ID and a protocol associated with the session.   
     
     
       48. The network monitoring device of claim 39, wherein for generating the common header by summarizing the first information, the processor further executes the computer-readable instructions to:
 extract static data from a header of the packet and the header of each of the at least one other packet; and   store the static data in the common header.   
     
     
       49. The network monitoring device of claim 39, wherein for generating the common header by summarizing the first information, the processor further executes the computer-readable instructions to summarize non-static data that is changed in the packet and the at least one other packet. 
     
     
       50. The network monitoring device of claim 39, wherein for generating the common header by summarizing the first information, the processor further executes the computer-readable instructions to extract the first information from the packet and each of the at least one other packet, and wherein the first information is based upon a protocol associated with the session. 
     
     
       51. The network monitoring device of claim 39, wherein for generating the payload metadata by summarizing the second information, the processor further executes the computer-readable instructions to:
 determine a data payload from the packet and each of the at least one other packet; and   store a subset of the data payload as the payload metadata, wherein the subset of the data payload is based on a protocol associated with the session.   
     
     
       52. A method comprising:
 determining, by a processor executing computer-readable instructions stored on a memory, a session associated with a packet received from a network;   generating, by the processor, a common header by summarizing first information of the packet and at least one other packet of the same session;   generating, by the processor, a payload metadata by summarizing second information of the packet and the at least one other packet by:
 determining, by the processor, a data payload from the packet and each of the at least one other packet; and 
 storing, by the processor, a subset of the data payload as the payload metadata, wherein the subset of the data payload is based on a protocol associated with the session; and 
   generating, by the processor, a session record comprising the common header and the payload metadata for evaluating at least one of the network, an application associated with the network, or a service associated with the network.   
     
     
       53. The method of claim 52, further comprising buffering, by the processor, data associated with the packet and the at least one other packet before generating the common header and the payload metadata. 
     
     
       54. The method of claim 52, further comprising determining, by the processor, the protocol associated with the session based upon information associated with the packet and the at least one other packet. 
     
     
       55. The method of claim 52, further comprising:
 assigning, by the processor, a new session ID to the session upon determining that the session is a new session or assigning an existing session ID to the session upon determining that the session is a pre-existing session, wherein the existing session ID is associated with the pre-existing session; and   generating, by the processor, session information based on the new session ID or the existing session ID and the protocol associated with the session.   
     
     
       56. The method of claim 52, wherein generating the common header by summarizing the first information further comprises:
 extracting, by the processor, static data from a header of the packet and the header of each of the at least one other packet; and   storing, by the processor, the static data in the common header.   
     
     
       57. The method of claim 52, wherein generating the common header by summarizing the first information further comprises summarizing, by the processor, non-static data that is changed in the packet and the at least one other packet. 
     
     
       58. The method of claim 52, wherein generating the common header by summarizing the first information further comprises extracting, by the processor, the first information from the packet and each of the at least one other packet, wherein the first information is based upon the protocol associated with the session. 
     
     
       59. The method of claim 52, wherein generating the common header and the payload metadata further comprises generating, by the processor, the common header and the payload metadata in real-time responsive to receiving the packet and the at least one other packet from the network. 
     
     
       60. The method of claim 52, further comprising:
 generating, by the processor, a performance parameter representing performance at least one of the network, the application, the service, a client associated with the network, or a server associated with the network, wherein the performance parameter is generated based on the packet and the at least one other packet; and   including, by the processor, the performance parameter in the session record.   
     
     
       61. The method of claim 52, further comprising generating, by the processor, mapping information based on the session and at least one other session, and wherein the mapping information is included in the session record.