Gap identification in corpora

Abstract

Embodiments of the present invention disclose a method, a computer program product, and a computer system for identifying information gaps in corpora. A computer receives a document and extracts keywords from the document while filtering trivial keywords. The computer identifies and extracts top keywords detailed by the document using a topic modelling approach before determining whether the extracted top keywords exceed a threshold use frequency. Based on determining that the top keywords exceed a threshold use frequency, determining whether the top keywords have a relation to other entities within the document and, if so, determining whether the top keywords are defined within the document. Based on determining that the top keywords are not defined in the document, adding the top keywords to a list and defining the top keywords.

What is claimed is: 
     
       1. A computer-implemented method of identifying information gaps in corpora, wherein the method comprises:
 a computer receiving a document comprising a handwritten portion from a user via a scanning interface of a computing device of the user; 
 the computer identifying one or more top keywords within the document, using a pixel analysis technique, wherein the one or more top keywords are the most common substantive words in the document; 
 the computer determining whether at least one top keyword of the one or more top keywords lacks a definition within the document by searching for language that matches a template; 
 based on determining that the at least one top keyword of the one or more top keywords lacks a definition within the document, and further based on determining, using one or more online databases and entity identification techniques, that the at least one top keyword is not any of a stop word, a function word, and a named entity, the computer adding the at least one top keyword to a list; and 
 based on adding the at least one top keyword to the list, the computer prompting the user via a user input for the definition of the at least one top keyword, given a context within the document, and the computer generating a knowledge graph corresponding to the document, wherein the knowledge graph identifies relations between the one or more top keywords, wherein the relations include whether the one or more top keywords are defined, undefined, and unrelated. 
 
     
     
       2. The method of  claim 1 , further comprising:
 the computer defining the at least one top keyword. 
 
     
     
       3. The method of  claim 1 , wherein adding the at least one top keyword to the list is further based on:
 the computer determining that the at least one top keyword has a relation to one or more entities within the document. 
 
     
     
       4. The method of  claim 3 , wherein determining that the at least one keyword has a relation to one or more entities within the document further comprises:
 the computer generating the knowledge graph corresponding to the document. 
 
     
     
       5. The method of  claim 1 , wherein adding the at least one top keyword to the list is further based on:
 the computer determining that the at least one top keyword is repeated within the document greater than a threshold number of times. 
 
     
     
       6. The method of  claim 1 , wherein identifying one or more top keywords within the document is performed using Latent Dirichlet Allocation topic modelling. 
     
     
       7. A computer program product for identifying information gaps in corpora, the computer program product comprising:
 one or more computer-readable storage media and program instructions stored on the one or more computer-readable storage media, the program instructions comprising: 
 program instructions to receive a document by a computer comprising a handwritten portion from a user via a scanning interface of a computing device of the user; 
 program instructions to identify one or more top keywords within the document, using a pixel analysis technique, wherein the one or more top keywords are the most common substantive words in the document; 
 program instructions to determine whether at least one top keyword of the one or more top keywords lacks a definition within the document; 
 based on determining that the at least one top keyword of the one or more top keywords lacks a definition within the document, and further based on determining, using one or more online databases and entity identification techniques, that the at least one top keyword is not any of a stop word, a function word, and a named entity, program instructions to add the at least one top keyword to a list; and 
 based on adding the at least one top keyword to the list, the computer prompting the user via a user input for the definition of the at least one top keyword, given a context within the document, and the computer generating a knowledge graph corresponding to the document, wherein the knowledge graph identifies relations between the one or more top keywords, wherein the relations include whether the one or more top keywords are defined, undefined, and unrelated. 
 
     
     
       8. The computer program product of  claim 7 , further comprising:
 program instructions to define the at least one top keyword. 
 
     
     
       9. The computer program product of  claim 7 , wherein adding the at least one top keyword to the list is further based on:
 program instructions to determine that the at least one top keyword has a relation to one or more entities within the document. 
 
     
     
       10. The computer program product of  claim 9 , wherein determining that the at least one keyword has a relation to one or more entities within the document further comprises:
 program instructions to generate the knowledge graph corresponding to the document. 
 
     
     
       11. The computer program product of  claim 7 , wherein adding the at least one top keyword to the list is further based on:
 program instructions to determine that the at least one top keyword is repeated within the document greater than a threshold number of times. 
 
     
     
       12. The computer program product of  claim 7 , wherein identifying one or more top keywords within the document is performed using Latent Dirichlet Allocation topic modelling. 
     
     
       13. A computer system for identifying information gaps in corpora, the computer system comprising:
 one or more computer processors, one or more computer-readable storage media, and program instructions stored on one or more of the computer-readable storage media for execution by at least one of the one or more processors, the program instructions comprising: 
 program instructions to receive a document by a computer comprising a handwritten portion from a user via a scanning interface of a computing device of the user; 
 program instructions to identify one or more top keywords within the document, using a pixel analysis technique, wherein the one or more top keywords are the most common substantive words in the document; 
 program instructions to determine whether at least one top keyword of the one or more top keywords lacks a definition within the document; 
 based on determining that the at least one top keyword of the one or more top keywords lacks a definition within the document, and further based on determining, using one or more online databases and entity identification techniques, that the at least one top keyword is not any of a stop word, a function word, and a named entity, program instructions to add the at least one top keyword to a list; and 
 based on adding the at least one top keyword to the list, the computer prompting the user via a user input for the definition of the at least one top keyword, given a context within the document, and the computer generating a knowledge graph corresponding to the document, wherein the knowledge graph identifies relations between the one or more top keywords, wherein the relations include whether the one or more top keywords are defined, undefined, and unrelated. 
 
     
     
       14. The computer system of  claim 13 , further comprising:
 program instructions to define the at least one top keyword. 
 
     
     
       15. The computer system of  claim 13 , wherein adding the at least one top keyword to the list is further based on:
 program instructions to determine that the at least one top keyword has a relation to one or more entities within the document. 
 
     
     
       16. The computer system of  claim 15 , wherein determining that the at least one keyword has a relation to one or more entities within the document further comprises:
 program instructions to generate the knowledge graph corresponding to the document. 
 
     
     
       17. The computer system of  claim 13 , wherein adding the at least one top keyword to the list is further based on:
 program instructions to determine that the at least one top keyword is repeated within the document greater than a threshold number of times.