Adaptive log analysis

Abstract

A method for obtaining information and status about a monitored system by adaptively analyzing log messages is provided. A log analyzer receives log messages generated by a monitored system. The log analyzer identifies static and variable portions in the received log messages. The log analyzer generates a template based on the identified static and variable portions of the received log messages. The log analyzer computes a metric for the generated template based on a number of log messages that fall within the template. The log analyzer reports a status in the monitored system based on the computed metric.

What is claimed is: 
     
       1. A computing device comprising:
 a processor; and 
 a storage device storing a set of instructions, wherein an execution of the set of instructions by the processor configures the computing device to perform acts comprising:
 receiving one or more log messages generated by a monitored system; 
 identifying static and variable portions in the received log messages; 
 generating a template based on the identified static and variable portions of the received one or more log messages; 
 computing a metric for the generated template based on a number of log messages of the one or more log messages that fall within the template; and 
 reporting a status in the monitored system based on the computed metric. 
 
 
     
     
       2. The computing device of  claim 1 , wherein the static and variable portions of the one or more log messages are identified by using a dictionary of meaningful words that are identified based on statistics of words appearing in log messages. 
     
     
       3. The computing device of  claim 1 , wherein the static and variable portions of the one or more log messages are identified by a list of words that co-occur in the log messages. 
     
     
       4. The computing device of  claim 1 , wherein an execution of the set of instructions by the processor further configures the computing device to perform an act comprising:
 determining a time frame of occurrence for the reported status based on a time stamp of a log message that falls within the template. 
 
     
     
       5. The computing device of  claim 1 , wherein the status of the monitored system is reported based on a template having a highest metric among a plurality of generated templates. 
     
     
       6. The computing device of  claim 1 , wherein the template is incrementally updated based on one or more subsequently received log messages. 
     
     
       7. The computing device of  claim 1 , wherein an execution of the set of instructions by the processor further configures the computing device to perform an act comprising:
 identifying a set of related log messages and identifying a set of templates that the set of related log messages fall within, as a template model. 
 
     
     
       8. The computing device of  claim 7 , wherein an execution of the set of instructions by the processor further configures the computing device to perform acts comprising:
 adding a particular template to the template model when one or more log messages of the set of related log messages fall within the particular template; and 
 removing a given particular template from the template model when no log messages of the set of related log messages fall within the given particular template. 
 
     
     
       9. The computing device of  claim 7 , wherein an execution of the set of instructions by the processor further configures the computing device to perform acts comprising:
 determining a time frame of occurrence for the reported status based on one or more time stamps in incoming log messages that fall within a template model that is related to the reported status. 
 
     
     
       10. The computing device of  claim 1 , wherein the metric is computed based on a ratio of number of log messages that fall within the template with respect to total number of log messages. 
     
     
       11. A computer program product comprising:
 one or more non-transitory computer-readable storage devices and program instructions stored on at least one of the one or more non-transitory storage devices, the program instructions executable by a processor, the program instructions comprising sets of instructions for:
 receiving one or more log messages generated by a monitored system; 
 identifying static and variable portions in the received log messages; 
 generating a template based on the identified static and variable portions of the received one or more log messages; 
 computing a metric for the generated template based on a number of log messages of the one or more log messages that fall within the template; and 
 reporting a status in the monitored system based on the computed metric. 
 
 
     
     
       12. A computer-implemented method comprising:
 receiving log messages generated by a monitored system; 
 identifying static and variable portions in the received log messages; 
 generating a template based on the identified static and variable portions of the received log messages; 
 computing a metric for the generated template based on a number of log messages of the one or more log messages that fall within the template; and 
 reporting a status in the monitored system based on the computed metric. 
 
     
     
       13. The computer-implemented method of  claim 12 , wherein the static and variable portions of the one or more log messages are identified by using a dictionary of meaningful words that are identified based on statistics of words appearing in log messages. 
     
     
       14. The computer-implemented method of  claim 12 , wherein the static and variable portions of the one or more log messages are identified by a list of words that co-occur in the log messages. 
     
     
       15. The computer-implemented method of  claim 12 , further comprising determining a time frame of occurrence for the reported status based on a time stamp of a log message that fall within the template. 
     
     
       16. The computer-implemented method of  claim 12 , wherein the template is incrementally updated based on one or more subsequently received log messages. 
     
     
       17. The computer-implemented method of  claim 12 , further comprising:
 identifying a set of related log messages; and 
 identifying a set of templates that the set of related log messages fall within as a template model. 
 
     
     
       18. The computer-implemented method of  claim 17 , further comprising adding a particular template to the template model when one or more log messages of the set of related log messages fall within the particular template. 
     
     
       19. The computer-implemented method of  claim 17 , further comprising removing a particular template from the template model when no log messages of the set of related log messages fall within the particular template. 
     
     
       20. The computer-implemented method of  claim 12 , wherein the metric is computed based on a ratio of a number of log messages that fall within the template with respect to a total number of log messages.