System and method for domain-independent terminology linking

Abstract

An automatic terminology linking system includes a candidate generator configured to identify candidate nodes for each terminology that is to be linked to a node of the knowledge base. A pseudo-candidate generator is configured to identify pseudo-candidate nodes for candidate-less terminologies. A candidate scorer is configured to respectively score the candidate nodes and the pseudo-candidate nodes by collective inference using occurrence statistics and co-occurrence statistics for these nodes. The pseudo-candidate generator is configured to identify knowledge base nodes that are semantically-related to candidate-less terminology as the pseudo-candidate nodes for the candidate-less terminology.

What is claimed is: 
     
       1. An automatic terminology linking system for automatically linking terminology to nodes of a knowledge base, the system comprising:
 a processor configured to:
 identify candidate nodes for each terminology that is to be linked to a node of the knowledge base using occurrence statistics, wherein each terminology determined to have no candidates is identified as candidate-less terminology; 
 identify pseudo-candidate nodes for the candidate-less terminology that is to be linked to a node of the knowledge base, the pseudo-candidate nodes being semantically-related to candidate-less terminology; 
 respectively score the candidate nodes and the pseudo-candidate nodes by collective inference using occurrence statistics and co-occurrence statistics for nodes of the knowledge base corresponding to the candidates and pseudo-candidates, 
 link each terminology to the node of the knowledge base corresponding to the highest-scored candidate node for the terminology; and 
 link each candidate-less terminology to the node of the knowledge base corresponding to the highest-scored pseudo-candidate for the candidate-less terminology, 
 wherein the occurrence statistics indicate probability that a terminology is linked to a respective node of the knowledge base, and 
 wherein the co-occurrence statistics indicate (i) a probability that two terminologies appear in a same document or (ii) a probability that two nodes of the knowledge based are linked to by a same document. 
 
 
     
     
       2. The system of  claim 1 , the processor being further configured to:
 determine the occurrence statistics and the co-occurrence statistics for the nodes of the knowledge base using a collection of documents. 
 
     
     
       3. The system of  claim 1 , the processor being further configured to:
 receive a document and a list of terminologies in the document as input. 
 
     
     
       4. The system of  claim 3 , the processor being further configured to:
 generate technical documents with terminologies linked to associated nodes of the knowledge base. 
 
     
     
       5. The system of  claim 1 , the processor being further configured to:
 identify knowledge base nodes that are near-synonyms to the candidate-less terminology as pseudo-candidates for the candidate-less terminology. 
 
     
     
       6. The system of  claim 1 , the processor being further configured to:
 identify knowledge base nodes that share words with the candidate-less terminology as pseudo-candidates for the candidate-less terminology. 
 
     
     
       7. The system of  claim 1 , the processor being further configured to:
 identify knowledge base nodes that are distributionally similar to the candidate-less terminology as pseudo-candidates for the candidate-less terminology. 
 
     
     
       8. The system of  claim 1 , the processor being further configured to:
 determine a distributional similarity between the candidate-less terminology and the nodes of the knowledge base using vector representations of the knowledge base nodes and the unlinked terminology. 
 
     
     
       9. A method for automatically linking terminology to nodes of a knowledge base, the method comprising:
 identifying candidate nodes for each terminology that is to be linked to a node of the knowledge base using occurrence statistics, wherein each terminology determined to have no candidates is identified as candidate-less terminology; 
 identifying pseudo-candidate nodes for the candidate-less terminology that is to be linked to a node of the knowledge base, the pseudo-candidate nodes being semantically-related to candidate-less terminology; 
 scoring the candidate nodes and the pseudo-candidate nodes by collective inference using occurrence statistics and co-occurrence statistics for nodes of the knowledge base; 
 linking each terminology to the highest-scored candidate for that terminology; and 
 linking each candidate-less terminology with the highest-scored pseudo-candidate for that candidate-less terminology, 
 wherein the occurrence statistics indicate probability that a terminology is linked to a respective node of the knowledge base, and 
 wherein the co-occurrence statistics indicate (i) a probability that two terminologies appear in a same document or (ii) a probability that two nodes of the knowledge based are linked to by a same document. 
 
     
     
       10. The method of  claim 9 , further comprising:
 determining the occurrence statistics and the co-occurrent statistics for the nodes of the knowledge base using a collection of documents. 
 
     
     
       11. The method of  claim 9 , further comprising:
 receiving a document and a list of terminologies in the document as input to identifying the candidate nodes. 
 
     
     
       12. The method of  claim 11 , further comprising:
 generating a technical document with terminologies linked to associated nodes of the knowledge base. 
 
     
     
       13. The method of  claim 9 , the identifying pseudo-candidate nodes further comprising:
 identifying knowledge base nodes that are near-synonyms to the candidate-less terminology as pseudo-candidates for the candidate-less terminology. 
 
     
     
       14. The method of  claim 9 , the identifying pseudo-candidate nodes further comprising:
 identifying knowledge base nodes that share words with the candidate-less terminology as pseudo-candidates for the candidate-less terminology. 
 
     
     
       15. The method of  claim 9 , the identifying pseudo-candidate nodes further comprising:
 identifying knowledge base nodes that are distributionally similar to the candidate-less terminology as pseudo-candidates for the candidate-less terminology. 
 
     
     
       16. The method of  claim 9 , further comprising:
 determining a distributional similarity between the candidate-less terminology and the nodes of the knowledge base using vector representations of the knowledge base nodes and the unlinked terminology.