Determining root-cause of failures based on machine-generated textual data

Abstract

A method and system for determining root-causes of incidences using machine-generated textual data. The method comprises receiving machine-generated textual data from at least one data source; classifying the received machine-generated textual data into at least one statistical metric; processing the statistical metric to recognize a plurality of incidence patterns; correlating the plurality of incidence patterns to identify at least a root-cause of an incidence that occurred in a monitored environment; and generating an alert indicating at least the identified root-cause.

What is claimed is: 
     
       1. A method implemented by a computer system for determining root-causes of incidences using machine-generated textual data, comprising:
 receiving, at the computer system, machine-generated textual data from at least one data source, at least some of the received data being unstructured data; 
 classifying, by the computer system, the received machine-generated textual data into at least one statistical metric, wherein classifying the machine-generated textual data into statistical metrics further comprises:
 grouping the machine-generated textual data into a plurality of events; 
 processing each event to determine a plurality of elements embedded therein; 
 determining a type of each of the plurality of elements; and 
 determining a statistical metric for each element based on at least the type of the element; 
 
 processing, by the computer system, the statistical metric to recognize a plurality of incidence patterns; 
 correlating, by the computer system, the plurality of incidence patterns to identify at least a root-cause of an incidence that occurred in a monitored environment; and 
 generating, by the computer system, an alert indicating at least the identified root-cause, wherein generating the alert further comprises grouping a plurality of alerts into one incident, the plurality of alerts having the identified root cause in common; 
 wherein the receiving, classifying, processing, correlating, and generating are performed without requiring any human interaction. 
 
     
     
       2. The method of  claim 1 , further comprising:
 selecting a set of incidence patterns from the plurality of recognized incidence patterns; and 
 correlating the selected set of selected incidence patterns. 
 
     
     
       3. The method of  claim 2 , wherein selecting the set of incidence patterns is based on at least one of: an amplitude of an incidence pattern, a frequency of an incidence pattern, a similarity of an incidence pattern to previously detected incidence patterns, and a number of detected same or similar incidence patterns. 
     
     
       4. The method of  claim 2 , wherein the set of selected incidence patterns include incidence patterns having at least one similar entity. 
     
     
       5. The method of  claim 4 , further comprising:
 identifying the at least one entity in a first incidence pattern; and 
 
       scanning a subset of the plurality of recognized incidence patterns to detect incidence patterns including the at least one entity, wherein the subset of the plurality of recognized incidence patterns occurred in a predefined time window prior to the first incidence pattern. 
     
     
       6. The method of  claim 1 , wherein each of the plurality of incidence patterns represents at least one of: a new behavior, an anomalous behavior, a routine operational change, a new trend, a changing trend, and an ongoing trend. 
     
     
       7. The method of  claim 1 , wherein correlating the plurality of incidence patterns to identify the at least a root-cause further comprises:
 determining the root-cause based on a correlation type being utilized. 
 
     
     
       8. The method of  claim 7 , wherein the correlation type is based on time-proximity. 
     
     
       9. The method of  claim 8 , further comprising:
 correlating at least two incidence patterns that occurred at the same or substantially the same time, wherein the root-cause is determined to be an incidence observed by an incidence pattern that occurred before other correlated incidence patterns. 
 
     
     
       10. The method of  claim 7 , wherein the correlation type is order-based. 
     
     
       11. The method of  claim 10 , further comprising:
 correlating at least two incidence patterns to identify at least one incidence pattern trended to at least an increased severity, wherein the root-cause is determined to be an incidence observed by the least one trended incidence pattern. 
 
     
     
       12. The method of  claim 7 , wherein the correlation type is component-based. 
     
     
       13. The method of  claim 12 , further comprising:
 correlating incidence patterns across different components to identify a component that includes a single broken element, wherein the root-cause is determined to be an incidence observed by an incidence pattern of the single broken element, wherein each of the different components includes a plurality of elements. 
 
     
     
       14. The method of  claim 1 , wherein each statistical metric is any one of: a gauge, a meter, and a histogram. 
     
     
       15. The method of  claim 1 , wherein the machine-generated textual data includes at least one of: application logs, configuration files, alerts, sensory signals, audit records, and combinations thereof. 
     
     
       16. The method of  claim 1 , wherein the monitored environment is an information technology (IT) infrastructure. 
     
     
       17. The method of  claim 1 , wherein determining the statistical metric for each element further comprises:
 determining a type of the statistical metric that allows for statistically measuring a value of the respective element. 
 
     
     
       18. A non-transitory computer readable medium having stored thereon instructions for causing a computer system to execute a process for determining cause root of incidences using machine-generated textual data, the process comprising the steps of:
 receiving at the computer system machine-generated textual data from at least one data source, at least some of the received data being unstructured data; 
 classifying, by the computer system, the received machine-generated textual data into at least one statistical metric, wherein classifying the machine-generated textual data into statistical metrics further comprises:
 grouping the machine-generated textual data into a plurality of events; 
 processing each event to determine a plurality of elements embedded therein; 
 determining a type of each of the plurality of elements; and 
 
 determining a statistical metric for each element based on at least the type of the element; 
 processing, by the computer system, the statistical metric to recognize a plurality of incidence patterns; 
 correlating, by the computer system, the plurality of incidence patterns to identify at least a root-cause of an incidence that occurred in a monitored environment; and 
 generating, by the computer system, an alert indicating at least the identified root-cause, wherein generating the alert includes grouping a plurality of alerts into one incident, the plurality of alerts having the identified root cause in common; 
 wherein the receiving, classifying, processing, correlating, and generating are performed without requiring any human interaction. 
 
     
     
       19. A system for determining root-causes of incidences using machine-generated textual data, comprising:
 a processing circuit; 
 a memory communicatively connected to the processing circuit, wherein the memory contains instructions that, when executed by the processing element, configure the processing circuit to: 
 receive at the system machine-generated textual data from at least one data source; 
 classify by the system the received machine-generated textual data into at least one statistical metric, wherein the system is further configured to:
 group the machine-generated textual data into a plurality of events; 
 process each event to determine a plurality of elements embedded therein; 
 determine a type of each of the plurality of elements; and 
 determine a statistical metric for each element based on at least the type of the element; 
 
 process by the system the statistical metric to recognize a plurality of incidence patterns; 
 correlate by the system the plurality of incidence patterns to identify at least a root-cause of an incidence that occurred in a monitored environment; and 
 generate by the system an alert indicating at least the identified root-cause, wherein the system is further configured to group a plurality of alerts into one incident, the plurality of alerts having the identified root cause in common; 
 wherein the system operates without requiring any human interaction. 
 
     
     
       20. The system of  claim 19 , wherein the method further configured to:
 select a set of incidence patterns from the plurality of recognized incidence patterns; and 
 correlate the selected set of selected incidence patterns. 
 
     
     
       21. The system of  claim 20 , wherein the selection of the set of incidence patterns is based on at least one of: an amplitude of an incidence pattern, a frequency of an incidence pattern, a similarity of an incidence pattern to previously detected incidence patterns, and a number of detected same or similar incidence patterns. 
     
     
       22. The system of  claim 21 , wherein the monitored environment is an information technology (IT) infrastructure. 
     
     
       23. The system of  claim 20 , wherein the machine-generated textual data includes at least one of: application logs, configuration files, alerts, sensory signals, audit records, and combinations thereof. 
     
     
       24. The system of  claim 19 , wherein each of the plurality of incidence patterns represents at least one of: a new behavior, an anomalous behavior, a routine operational change, a new trend, a changing trend, and an ongoing trend. 
     
     
       25. The system of  claim 19 , wherein correlating the system is further configured to:
 determine the root-cause based on a correlation type being utilized. 
 
     
     
       26. The system of  claim 25 , wherein the correlation type is based on time-proximity. 
     
     
       27. The system of  claim 26 , wherein the system is further configured to:
 correlate at least two incidence patterns that occurred at the same or substantially the same time, wherein the root-cause is determined to be an incidence observed by an incidence pattern that occurred before other correlated incidence patterns. 
 
     
     
       28. The system of  claim 25 , wherein the correlation type is order-based. 
     
     
       29. The system of  claim 28 , wherein the system is further configured to:
 correlate at least two incidence patterns to identify at least one incidence pattern trended to at least an increased severity, wherein the root-cause is determined to be an incidence observed by the least one trended incidence pattern. 
 
     
     
       30. The system of  claim 25 , wherein the correlation type is component-based. 
     
     
       31. The system of  claim 30 , wherein the system is further configured to:
 correlate incidence patterns across different components to identify a component that includes a single broken element, wherein the root-cause is determined to be an incidence observed by an incidence pattern of the single broken element, wherein each of the different components includes a plurality of elements. 
 
     
     
       32. The system of  claim 19 , wherein each statistical metric is any one of: a gauge, a meter, and a histogram.