Method of automated discovery of new topics

Abstract

The present disclosure relates to a method for performing automated discovery of new topics from unlimited documents related to any subject domain, employing a multi-component extension of Latent Dirichlet Allocation (MC-LDA) topic models, to discover related topics in a corpus. The resulting data may contain millions of term vectors from any subject domain identifying the most distinguished co-occurring topics that users may be interested in, for periodically building new topic ID models using new content, which may be employed to compare one by one with existing model to measure the significance of changes, using term vectors differences with no correlation with a Periodic New Model, for periodic updates of automated discovery of new topics, which may be used to build a new topic ID model in-memory database to allow query-time linking on massive data-set for automated discovery of new topics.

What is claimed is: 
     
         1 . A computer-implemented method comprising:
 extracting, by a computer, from a document corpus, data associated with a plurality of co-occurring topics;   in response to extracting the data associated with the plurality of co-occurring topics, extracting, by the computer, a plurality of topic identifiers from the plurality of co-occurring topics;   generating, by the computer, a periodic topic model comprising a set of one or more term vectors by comparing topic significance among the plurality of topic identifiers;   periodically creating, by the computer, new topic ID models using data content in the periodic topic model by identifying a similarity of topics;   generating, by the computer, fuzzy keys for the new topic ID models; and   indexing, by the computer, data in the new topic ID models based on the fuzzy keys for non-exclusionary searching and automated discovery of new topics.   
     
     
         2 . The computer-implemented method of  claim 1 , further comprising identifying, by the computer, in one or more document corpora of a data source, a topic of interest based upon one or more concurring topics identified in the one or more document corpora. 
     
     
         3 . The computer-implemented method of  claim 2 , further comprising automatically extracting, by the computer, from the document corpus, data associated with the plurality of co-occurring topics based on the topic of interest. 
     
     
         4 . The computer-implemented method of  claim 1 , further comprising determining, by the computer, a relationship of corresponding term vectors from the plurality of co-occurring topics, each co-occurring topic of the plurality of co-occurring topics containing one or more term vectors. 
     
     
         5 . The computer-implemented method of  claim 4 , further comprising generating, by the computer, a master topic computer model comprising a first set of one or more term vectors identified in text of the document corpus upon determining the relationship of the corresponding term vectors from the plurality of co-occurring topics. 
     
     
         6 . The computer-implemented method of  claim 5 , further comprising selecting, by the computer, one or more new topics by identifying one or more term vectors from the set of the one or more term vectors in the periodic new topic computer model that has no correlation with the first set of one or more term vectors in the master topic computer model. 
     
     
         7 . The computer-implemented method of  claim 5 , further comprising adding, via the computer, the one or more new topics to the master topic computer model. 
     
     
         8 . The computer-implemented method of  claim 1 , wherein comparing the topic significance among the plurality of topic identifiers is based on a predetermined significance threshold. 
     
     
         9 . The computer-implemented method of  claim 5 , wherein the master topic computer model is a multi-component extension of a Latent Dirichlet Allocation (MC-LDA) topic model. 
     
     
         10 . The computer-implemented method of  claim 1 , wherein the periodic new topic computer model is a multi-component extension of a Latent Dirichlet Allocation (MC-LDA) topic model. 
     
     
         11 . The computer-implemented method of  claim 1 , wherein the set of the one or more term vectors in the periodic new topic computer model corresponds to a second set of the one or more term vectors. 
     
     
         12 . A system comprising:
 a database source computer module configured to store a document corpus; and   one or more computers comprising one or more processors configured to:
 extract from the document corpus, data associated with a plurality of co-occurring topics; 
 extract a plurality of topic identifiers from the plurality of co-occurring topics in response to extracting the data associated with the plurality of co-occurring topics; 
 generate a periodic topic model comprising a set of one or more term vectors by comparing topic significance among the plurality of topic identifiers; 
 periodically create new topic ID models using data content in the periodic topic model by identifying a similarity of topics; 
 generate fuzzy keys for the new topic ID models; and 
 index data in the new topic ID models based on the fuzzy keys for non-exclusionary searching and automated discovery of new topics. 
   
     
     
         13 . The system of  claim 12 , wherein the one or more computers are further configured to identify in one or more document corpora of a data source, a topic of interest based upon one or more concurring topics identified in the one or more document corpora. 
     
     
         14 . The system of  claim 13 , wherein the one or more computers are further configured to automatically extract from the document corpus, data associated with the plurality of co-occurring topics based on the topic of interest. 
     
     
         15 . The system of  claim 12 , wherein the one or more computers are further configured to determine a relationship of corresponding term vectors from the plurality of co-occurring topics, each co-occurring topic of the plurality of co-occurring topics containing one or more term vectors. 
     
     
         16 . The system of  claim 15 , wherein the one or more computers are further configured to generate a master topic computer model comprising a first set of one or more term vectors identified in text of the document corpus upon determining the relationship of the corresponding term vectors from the plurality of co-occurring topics. 
     
     
         17 . The system of  claim 16 , wherein the one or more computers are further configured to select one or more new topics by identifying one or more term vectors from the set of the one or more term vectors in the periodic new topic computer model that has no correlation with the first set of one or more term vectors in the master topic computer model. 
     
     
         18 . The system of  claim 16 , wherein the one or more computers are further configured to add the one or more new topics to the master topic computer model. 
     
     
         19 . The system of  claim 12 , wherein comparing the topic significance among the plurality of topic identifiers is based on a predetermined significance threshold. 
     
     
         20 . The system of  claim 16 , wherein the master topic computer model is a multi-component extension of a Latent Dirichlet Allocation (MC-LDA) topic model, and wherein the periodic new topic computer model is a multi-component extension of a Latent Dirichlet Allocation (MC-LDA) topic model.