Generating a log parser by automatically identifying regular expressions matching a sample log

Abstract

An approach is presented for generating a log parser. Regular expressions are received and stored in a crowd-sourced data repository. An instruction is received to create a log parser based on a sample log. The sample log is received. Matches are identified between strings of characters included in the received sample log and regular expressions included in the stored regular expressions. Each match indicates a stored regular expression is capable of parsing a string included in the sample log. Based on the identified matches, the log parser is generated so as to include the regular expressions that match the strings included in the sample log.

What is claimed is: 
     
         1 . A method of generating a log parser, the method comprising the steps of:
 a computer receiving regular expressions and storing the regular expressions in a crowd-sourced data repository;   subsequent to receiving and storing the regular expressions, the computer receiving an instruction to create a log parser based on a sample log;   the computer receiving the sample log;   based on the stored regular expressions and the received sample log, the computer identifying matches between a plurality of strings of characters included in the received sample log and a plurality of regular expressions included in the stored regular expressions, each match indicating a regular expression included in the plurality of regular expressions is capable of parsing a respective string included in the plurality of strings; and   based on the identified matches, the computer generating the log parser as including the plurality of regular expressions that match the plurality of strings included in the sample log.   
     
     
         2 . The method of  claim 1 , further comprising the steps of:
 the computer querying the data repository to attempt to identify one or more regular expressions in the data repository capable of parsing one string included in the sample log;   in response to the step of querying the data repository, the computer determining no regular expression in the data repository is capable of parsing the one string included in the sample log; and   subsequent to the step of determining no regular expression in the data repository is capable of parsing the one string included in the sample log, the computer receiving a user input of a new regular expression capable of parsing the one string included in the sample log, wherein the step of generating the log parser includes the step of generating the log parser as further including the new regular expression.   
     
     
         3 . The method of  claim 2 , further comprising the steps of:
 the computer storing the new regular expression in the data repository;   the computer receiving another instruction to create another log parser based on another sample log;   the computer receiving the other sample log;   based on the stored new regular expression and the received other sample log, the computer identifying a match between another string included in the received other sample log and the stored new regular expression; and   based on the identified match and based on the new regular expression being included in the generated log parser, the computer generating the other log parser as including the new regular expression that matches the other string included in the other sample log, without requiring another user input of the new regular expression.   
     
     
         4 . The method of  claim 1 , further comprising the steps of:
 the computer querying the data repository;   in response to the step of querying the data repository, the computer identifying one regular expression in the data repository that potentially matches one string included in the sample log;   the computer presenting a suggestion to use the identified one regular expression to parse the one string included in the sample log;   in response to the step of presenting the suggestion, the computer receiving a first user input rejecting the suggestion;   subsequent to the step of receiving the first user input rejecting the suggestion, the computer receiving a second user input of a new regular expression capable of parsing the one string included in the sample log, wherein the step of generating the log parser includes the step of generating the log parser as further including the new regular expression.   
     
     
         5 . The method of  claim 4 , further comprising the steps of:
 the computer storing the new regular expression in the data repository;   the computer receiving another instruction to create another log parser based on another sample log;   the computer receiving the other sample log;   based on the stored new regular expression and the received other sample log, the computer identifying a match between another string included in the received other sample log and the stored new regular expression; and   based on the identified match and based on the new regular expression being included in the generated log parser, the computer generating the other log parser as including the new regular expression that matches the other string included in the other sample log, without requiring another user input of the new regular expression.   
     
     
         6 . A computer system comprising:
 a central processing unit (CPU);   a memory coupled to the CPU;   a computer-readable, tangible storage device coupled to the CPU, the storage device containing instructions that, when carried out by the CPU via the memory, implement a method of generating a log parser, the method comprising the steps of:
 the computer system receiving regular expressions and storing the regular expressions in a crowd-sourced data repository; 
 subsequent to receiving and storing the regular expressions, the computer system receiving an instruction to create a log parser based on a sample log; 
 the computer system receiving the sample log; 
 based on the stored regular expressions and the received sample log, the computer system identifying matches between a plurality of strings of characters included in the received sample log and a plurality of regular expressions included in the stored regular expressions, each match indicating a regular expression included in the plurality of regular expressions is capable of parsing a respective string included in the plurality of strings; and 
 based on the identified matches, the computer system generating the log parser as including the plurality of regular expressions that match the plurality of strings included in the sample log. 
   
     
     
         7 . The computer system of  claim 6 , wherein the method further comprises the steps of:
 the computer system querying the data repository to attempt to identify one or more regular expressions in the data repository capable of parsing one string included in the sample log;   in response to the step of querying the data repository, the computer system determining no regular expression in the data repository is capable of parsing the one string included in the sample log; and   subsequent to the step of determining no regular expression in the data repository is capable of parsing the one string included in the sample log, the computer system receiving a user input of a new regular expression capable of parsing the one string included in the sample log, wherein the step of generating the log parser includes the step of generating the log parser as further including the new regular expression.   
     
     
         8 . The computer system of  claim 7 , wherein the method further comprises the steps of:
 the computer system storing the new regular expression in the data repository;   the computer system receiving another instruction to create another log parser based on another sample log;   the computer system receiving the other sample log;   based on the stored new regular expression and the received other sample log, the computer system identifying a match between another string included in the received other sample log and the stored new regular expression; and   based on the identified match and based on the new regular expression being included in the generated log parser, the computer system generating the other log parser as including the new regular expression that matches the other string included in the other sample log, without requiring another user input of the new regular expression.   
     
     
         9 . The computer system of  claim 6 , wherein the method further comprises the steps of:
 the computer system querying the data repository;   in response to the step of querying the data repository, the computer system identifying one regular expression in the data repository that potentially matches one string included in the sample log;   the computer system presenting a suggestion to use the identified one regular expression to parse the one string included in the sample log;   in response to the step of presenting the suggestion, the computer system receiving a first user input rejecting the suggestion;   subsequent to the step of receiving the first user input rejecting the suggestion, the computer system receiving a second user input of a new regular expression capable of parsing the one string included in the sample log, wherein the step of generating the log parser includes the step of generating the log parser as further including the new regular expression.   
     
     
         10 . The computer system of  claim 9 , wherein the method further comprises the steps of:
 the computer system storing the new regular expression in the data repository;   the computer system receiving another instruction to create another log parser based on another sample log;   the computer system receiving the other sample log;   based on the stored new regular expression and the received other sample log, the computer system identifying a match between another string included in the received other sample log and the stored new regular expression; and   based on the identified match and based on the new regular expression being included in the generated log parser, the computer system generating the other log parser as including the new regular expression that matches the other string included in the other sample log, without requiring another user input of the new regular expression.   
     
     
         11 . A computer program product comprising:
 a computer-readable, tangible storage device; and   computer-readable program instructions stored in the computer-readable, tangible storage device, the computer-readable program instructions, when carried out by a central processing unit (CPU) of a computer system, implement a method of generating a log parser, the method comprising the steps of:
 the computer system receiving regular expressions and storing the regular expressions in a crowd-sourced data repository; 
 subsequent to receiving and storing the regular expressions, the computer system receiving an instruction to create a log parser based on a sample log; 
 the computer system receiving the sample log; 
 based on the stored regular expressions and the received sample log, the computer system identifying matches between a plurality of strings of characters included in the received sample log and a plurality of regular expressions included in the stored regular expressions, each match indicating a regular expression included in the plurality of regular expressions is capable of parsing a respective string included in the plurality of strings; and 
 based on the identified matches, the computer system generating the log parser as including the plurality of regular expressions that match the plurality of strings included in the sample log. 
   
     
     
         12 . The computer program product of  claim 11 , wherein the method further comprises the steps of:
 the computer system querying the data repository to attempt to identify one or more regular expressions in the data repository capable of parsing one string included in the sample log;   in response to the step of querying the data repository, the computer system determining no regular expression in the data repository is capable of parsing the one string included in the sample log; and   subsequent to the step of determining no regular expression in the data repository is capable of parsing the one string included in the sample log, the computer system receiving a user input of a new regular expression capable of parsing the one string included in the sample log, wherein the step of generating the log parser includes the step of generating the log parser as further including the new regular expression.   
     
     
         13 . The computer program product of  claim 12 , wherein the method further comprises the steps of:
 the computer system storing the new regular expression in the data repository;   the computer system receiving another instruction to create another log parser based on another sample log;   the computer system receiving the other sample log;   based on the stored new regular expression and the received other sample log, the computer system identifying a match between another string included in the received other sample log and the stored new regular expression; and   based on the identified match and based on the new regular expression being included in the generated log parser, the computer system generating the other log parser as including the new regular expression that matches the other string included in the other sample log, without requiring another user input of the new regular expression.   
     
     
         14 . The computer program product of  claim 11 , wherein the method further comprises the steps of:
 the computer system querying the data repository;   in response to the step of querying the data repository, the computer system identifying one regular expression in the data repository that potentially matches one string included in the sample log;   the computer system presenting a suggestion to use the identified one regular expression to parse the one string included in the sample log;   in response to the step of presenting the suggestion, the computer system receiving a first user input rejecting the suggestion;   subsequent to the step of receiving the first user input rejecting the suggestion, the computer system receiving a second user input of a new regular expression capable of parsing the one string included in the sample log, wherein the step of generating the log parser includes the step of generating the log parser as further including the new regular expression.   
     
     
         15 . The computer program product of  claim 14 , wherein the method further comprises the steps of:
 the computer system storing the new regular expression in the data repository;   the computer system receiving another instruction to create another log parser based on another sample log;   the computer system receiving the other sample log;   based on the stored new regular expression and the received other sample log, the computer system identifying a match between another string included in the received other sample log and the stored new regular expression; and   based on the identified match and based on the new regular expression being included in the generated log parser, the computer system generating the other log parser as including the new regular expression that matches the other string included in the other sample log, without requiring another user input of the new regular expression.   
     
     
         16 . A process for supporting computing infrastructure, the process comprising:
 a first computer system providing at least one support service for at least one of creating, integrating, hosting, maintaining, and deploying computer-readable code in a second computer system, the computer-readable code containing instructions, wherein the instructions, when carried out by a processor of the second computer system, implement a method of generating a log parser, the method comprising the steps of:
 the second computer system receiving regular expressions and storing the regular expressions in a crowd-sourced data repository; 
 subsequent to receiving and storing the regular expressions, the second computer system receiving an instruction to create a log parser based on a sample log; 
 the second computer system receiving the sample log; 
 based on the stored regular expressions and the received sample log, the second computer system identifying matches between a plurality of strings of characters included in the received sample log and a plurality of regular expressions included in the stored regular expressions, each match indicating a regular expression included in the plurality of regular expressions is capable of parsing a respective string included in the plurality of strings; and 
 based on the identified matches, the second computer system generating the log parser as including the plurality of regular expressions that match the plurality of strings included in the sample log. 
   
     
     
         17 . The process of  claim 16 , wherein the method further comprises the steps of:
 the second computer system querying the data repository to attempt to identify one or more regular expressions in the data repository capable of parsing one string included in the sample log;   in response to the step of querying the data repository, the second computer system determining no regular expression in the data repository is capable of parsing the one string included in the sample log; and   subsequent to the step of determining no regular expression in the data repository is capable of parsing the one string included in the sample log, the second computer system receiving a user input of a new regular expression capable of parsing the one string included in the sample log, wherein the step of generating the log parser includes the step of generating the log parser as further including the new regular expression.   
     
     
         18 . The process of  claim 17 , wherein the method further comprises the steps of:
 the second computer system storing the new regular expression in the data repository;   the second computer system receiving another instruction to create another log parser based on another sample log;   the second computer system receiving the other sample log;   based on the stored new regular expression and the received other sample log, the second computer system identifying a match between another string included in the received other sample log and the stored new regular expression; and   based on the identified match and based on the new regular expression being included in the generated log parser, the second computer system generating the other log parser as including the new regular expression that matches the other string included in the other sample log, without requiring another user input of the new regular expression.   
     
     
         19 . The process of  claim 16 , wherein the method further comprises the steps of:
 the second computer system querying the data repository;   in response to the step of querying the data repository, the second computer system identifying one regular expression in the data repository that potentially matches one string included in the sample log;   the second computer system presenting a suggestion to use the identified one regular expression to parse the one string included in the sample log;   in response to the step of presenting the suggestion, the second computer system receiving a first user input rejecting the suggestion;   subsequent to the step of receiving the first user input rejecting the suggestion, the second computer system receiving a second user input of a new regular expression capable of parsing the one string included in the sample log, wherein the step of generating the log parser includes the step of generating the log parser as further including the new regular expression.   
     
     
         20 . The process of  claim 19 , wherein the method further comprises the steps of:
 the second computer system storing the new regular expression in the data repository;   the second computer system receiving another instruction to create another log parser based on another sample log;   the second computer system receiving the other sample log;   based on the stored new regular expression and the received other sample log, the second computer system identifying a match between another string included in the received other sample log and the stored new regular expression; and   based on the identified match and based on the new regular expression being included in the generated log parser, the second computer system generating the other log parser as including the new regular expression that matches the other string included in the other sample log, without requiring another user input of the new regular expression.