Dictionary refinement for information extraction

Abstract

A method for refining a dictionary for information extraction, the operations including: inputting a set of extracted results from execution of an extractor comprising the dictionary on a collection of text, wherein the extracted results are labeled as correct results or incorrect results; processing the extracted results using an algorithm configured to set a score of the extractor above a score threshold, wherein the score threshold balances a precision and a recall of the extractor; and outputting a set of candidate dictionary entries corresponding to a full set of dictionary entries, wherein the candidate dictionary entries are candidates to be removed from the dictionary based on the extracted results.

1 . A computer program product, comprising:
 a computer readable storage medium to store a computer readable program, wherein the computer readable program, when executed by a processor within a computer, causes the computer to perform operations for refining a dictionary for information extraction, the operations comprising:
 inputting a set of extracted results from execution of an extractor matching the dictionary to a collection of text, wherein the extracted results are labeled as correct results or incorrect results; 
 processing the extracted results using an algorithm configured to set a score of the extractor above a score threshold, computed as a harmonic mean of a recall of the extractor and a precision of each dictionary entry, wherein the recall comprises a fraction of true positives among a total number of expected occurrences, wherein the precision comprises a probability that an extracted entity is correct; and 
 outputting a set of candidate dictionary entries corresponding to a full set of dictionary entries, wherein the candidate dictionary entries are candidates to be removed from the dictionary based on the extracted results. 
   
     
     
         2 . The computer program product of  claim 1 , wherein the computer readable program, when executed on the computer, causes the computer to perform additional operations, comprising:
 obtaining extracted results from the extractor using a plurality of specialized dictionaries, wherein each dictionary produces extracted results labeled as correct results or incorrect results.   
     
     
         3 . The computer program product of  claim 1 , wherein inputting a set of extracted results further comprises:
 using the extractor comprising a set of predetermined rules and matching a plurality of dictionaries to the collection of text to determine the extracted results; and   labeling the correct results and the incorrect results based on a user input.   
     
     
         4 . The computer program product of  claim 1 , wherein processing the extracted results further comprises:
 computing the set of candidate dictionary entries that set the score above the score threshold, wherein the set of candidate dictionary entries comprises a maximum size constraint.   
     
     
         5 . The computer program product of  claim 1 , wherein processing the extracted results further comprises:
 computing the set of candidate dictionary entries that set the score above the score threshold within an allocated recall constraint, wherein the recall constraint determines a minimum coverage of the dictionary.   
     
     
         6 . The computer program product of  claim 1 , wherein processing the extracted results further comprises:
 estimating the precision of each dictionary entry in the full set of dictionary entries using the extracted results, wherein the algorithm is a statistical estimation algorithm.   
     
     
         7 . The computer program product of  claim 6 , wherein the algorithm is an expectation-maximization (EM) algorithm. 
     
     
         8 - 14 . (canceled) 
     
     
         15 . A dictionary refinement system, comprising:
 an extractor configured to match a dictionary to a collection of text to obtain a set of extracted results, wherein the extracted results are labeled as correct results or incorrect results;   a processor configured to:
 process the extracted results using an algorithm configured to set a score of the extractor above a score threshold computed as a harmonic mean of a recall of the extractor and a precision of each dictionary entry, wherein the recall comprises a fraction of true positives among a total number of expected occurrences, wherein the precision comprises a probability that an extracted entity is correct; and 
 output a set of candidate dictionary entries corresponding to a full set of dictionary entries, wherein the candidate dictionary entries are candidates to be removed from the dictionary based on the extracted results. 
   
     
     
         16 . The system of  claim 15 , wherein the extractor is further configured to:
 obtain extracted results from the extractor using a plurality of specialized dictionaries, wherein the extracted results corresponding to each dictionary are labeled as correct results or incorrect results.   
     
     
         17 . The system of  claim 15 , wherein inputting a set of extracted results further comprises:
 using the extractor comprising a set of predetermined rules and matching a plurality of dictionaries to the collection of text to determine the extracted results; and   labeling the correct results and the incorrect results based on a user input.   
     
     
         18 . The system of  claim 15 , wherein processing the extracted results further comprises:
 computing the set of candidate dictionary entries that set the score above the score threshold, wherein the set of candidate dictionary entries comprises a maximum size constraint.   
     
     
         19 . The system of  claim 15 , wherein processing the extracted results further comprises:
 computing the set of candidate dictionary entries that set the score above the score threshold within an allocated recall constraint, wherein the recall constraint determines a minimum coverage of the dictionary.   
     
     
         20 . The system of  claim 15 , wherein the extractor is further configured to:
 estimate the precision of each dictionary entry in the full set of dictionary entries using the extracted results, wherein the algorithm is a statistical estimation algorithm.