Method of automated discovery of topic relatedness

Abstract

A computer system and method for automated discovery of topic relatedness are disclosed. According to an embodiment, topics within documents from a corpus may be discovered by applying multiple topic identification (ID) models, such as multi-component latent Dirichlet allocation (MC-LDA) or similar methods. Each topic model may differ in a number of topics. Discovered topics may be linked to the associated document. Relatedness between discovered topics may be determined by analyzing co-occurring topic IDs from the different models, assigning topic relatedness scores, where related topics may be used for matching/linking a feature of interest. The disclosed method may have an increased disambiguation precision, and may allow the matching and linking of documents using the discovered relationships.

What is claimed is: 
     
         1 . A method comprising:
 in response to generating, by a processor, a first term vector defining a first topic in a set of documents based on a first topic model and a second term vector defining a second topic in the set of documents based on a second topic model,
 identifying, by the processor, a third term vector defining a third topic in the set of documents and 
 linking, by the processor, the first term vector, the second term vector, and the third term vector with the set of documents, wherein the first topic model and the second topic model are formed based on a set of data extracted from the set of documents, 
 wherein the first topic model is implemented based on a generative statistical model with a first set of parameters, 
 wherein the second topic model is implemented based on the generative statistical model with a second set of parameters, 
 wherein the first set of parameters is distinct from the second set of parameters, and 
 wherein the third topic is not represented by the first topic and the second topic in the set of documents; 
   in response to comparing, by the processor, the first term vector, the second term vector, and the third term vector across the set of documents,
 assigning, by the processor, a relatedness score to each of the first term vector, the second term vector, and the third term vector based on co-occurrence of each of the first term vector, the second term vector, and the third term vector across the set of documents, and 
 determining, by the processor, whether the first term vector, the second term vector, and the third term vector are topic related across the set of documents based on the relatedness score. 
   
     
     
         2 . The method of  claim 1 , wherein the first model is based a multi-component extension of the generative statistical model. 
     
     
         3 . The method of  claim 2 , wherein the generative statistical model includes a latent Dirichlet allocation. 
     
     
         4 . The method of  claim 1 , wherein the second model is based on a multi-component extension of the generative statistical model. 
     
     
         5 . The method of  claim 4 , wherein the generative statistical model includes a latent Dirichlet allocation. 
     
     
         6 . The method of  claim 1 , further comprising:
 generating, by the processor, a hierarchy of related topics in the document corpus, wherein the hierarchy of related topics is based on the first term vector, the second term vector, and the third term vector.   
     
     
         7 . The method of  claim 1 , wherein the linking further comprises generating, by the processor, a graphical representation of co-occurring linked topics across the documents based on the first term vector, the second term vector, and the third term vector. 
     
     
         8 . The method of  claim 1 , wherein the first set of parameters includes at least one of a multi-document component, a vocabulary size, or a parameter setting for a prior Dirichlet distribution on a topic term. 
     
     
         9 . The method of  claim 1 , wherein the second set of parameters includes at least one of a multi-document component, a vocabulary size, or a parameter setting for a prior Dirichlet distribution on a topic term. 
     
     
         10 . The method of  claim 1 , further comprising:
 adding, by the processor, the third term vector to the first set of parameters after the determining.   
     
     
         11 . A system comprising:
 a processor and a memory, wherein the memory stores a set of instructions executable via the processor, wherein the set of instructions enables:
 in response to generating, by the processor, a first term vector defining a first topic in a set of documents based on a first topic model and a second term vector defining a second topic in the set of documents based on a second topic model, identifying, by the processor, a third term vector defining a third topic in the set of documents and linking, by the processor, the first term vector, the second term vector, and the third term vector with the set of documents, wherein the first topic model and the second topic model are formed based on a set of data extracted from the set of documents, wherein the first topic model is implemented based on a generative statistical model with a first set of parameters, wherein the second topic model is implemented based on the generative statistical model with a second set of parameters, wherein the first set of parameters is distinct from the second set of parameters, wherein the third topic is not represented by the first topic and the second topic in the set of documents; 
 in response to comparing, by the processor, the first term vector, the second term vector, and the third term vector across the set of documents, assigning, by the processor, a relatedness score to each of the first term vector, the second term vector, and the third term vector based on co-occurrence of each of the first term vector, the second term vector, and the third term vector across the set of documents, and determining, by the processor, whether the first term vector, the second term vector, and the third term vector are topic related across the set of documents based on the relatedness score. 
   
     
     
         12 . The system of  claim 11 , wherein the first model is based a multi-component extension of the generative statistical model. 
     
     
         13 . The system of  claim 12 , wherein the generative statistical model includes a latent Dirichlet allocation. 
     
     
         14 . The system of  claim 11 , wherein the second model is based on a multi-component extension of the generative statistical model. 
     
     
         15 . The system of  claim 14 , wherein the generative statistical model includes a latent Dirichlet allocation. 
     
     
         16 . The system of  claim 11 , wherein the method further comprises:
 generating, by the processor, a hierarchy of related topics in the document corpus, wherein the hierarchy of related topics is based on the first term vector, the second term vector, and the third term vector.   
     
     
         17 . The system of  claim 11 , wherein the linking further comprises generating, by the processor, a graphical representation of co-occurring linked topics across the documents based on the first term vector, the second term vector, and the third term vector. 
     
     
         18 . The system of  claim 11 , wherein the first set of parameters includes at least one of a multi-document component, a vocabulary size, or a parameter setting for a prior Dirichlet distribution on a topic term. 
     
     
         19 . The system of  claim 11 , wherein the second set of parameters includes at least one of a multi-document component, a vocabulary size, or a parameter setting for a prior Dirichlet distribution on a topic term. 
     
     
         20 . The system of  claim 11 , wherein the method further comprises:
 adding, by the processor, the third term vector to the first set of parameters after the determining.