Parallel-hierarchical model for machine comprehension on small data

Abstract

Examples of the present disclosure provide systems and methods relating to a machine comprehension test with a learning-based approach, harnessing neural networks arranged in a parallel hierarchy. This parallel hierarchy enables the model to compare the passage, question, and answer from a variety of perspectives, as opposed to using a manually designed set of features. Perspectives may range from the word level to sentence fragments to sequences of sentences, and networks operate on word-embedding representations of text. A training methodology for small data is also provided.

What is claimed is: 
     
       1. A system comprising:
 at least one processor; and 
 memory storing instructions that, when executed by the at least one processor, perform a set of operations comprising: 
 receiving input text, wherein the input text comprises text data, a question, and an answer candidate for the question based on the text data; 
 converting the input text into a word-by-word representation based at least on a word-by-word sentential match between a combination of the natural language question and the answer candidate and the text data; 
 converting the input text into a semantic representation of the input text based at least on a semantic match between the combination of the question and the answer candidate and the text data; 
 concurrently processing:
 generating a first result based on analyzing the word-by-word representation using one or more word-by-word processes, wherein each word-by-word process generates a first matching score indicating a degree of the word-by-word sentential match; and 
 generating a second result based on analyzing the semantic representation using semantic processes, wherein each semantic process generates a second matching score indicating a degree of the semantic match; 
 
 combining the first result and the second result; 
 determining, based on the combined first and second results, a top result, wherein the top result includes the combination of the question and the answer candidate corresponding to the highest score among a set of matching scores including the first matching score and the second matching score; and 
 providing the top result. 
 
     
     
       2. The system of  claim 1 , wherein each word-by-word processes use a multilayer perceptron neural network including the natural language question and the answer candidate,
 wherein the semantic processes use a multilayer perceptron plus sum neural network based on semantics of the input text, and 
 wherein the first result and the second result are distinct. 
 
     
     
       3. The system of  claim 1 , wherein the input text comprises a natural language question. 
     
     
       4. The system of  claim 1 , wherein the concurrently processing comprises a semantic analysis of the input text and a word-by-word analysis of the input text. 
     
     
       5. The system of  claim 4 , wherein the semantic analysis includes comparing a hypothesis to sentences in the text data. 
     
     
       6. The system of  claim 5 , wherein the hypothesis includes a combination of at least a portion of the question with at least a portion of the answer candidate. 
     
     
       7. The system of  claim 4 , wherein the word-by-word process comprises at least one of:
 a sentential process; 
 a sliding window sequential process; and 
 a dependency sliding window dependency process. 
 
     
     
       8. The system of  claim 7 , wherein the sliding window sequential process scans over words of the text data as one continuous sequence. 
     
     
       9. The system of  claim 7 , wherein the dependency sliding window dependency process comprises:
 constructing a dependency graph for a sentence in the text data; 
 reordering words in the sentence based at least in part on the dependency graph to generate a reordered sentence; and 
 scans over words of the reordered sentence. 
 
     
     
       10. The system of  claim 1 , wherein the concurrently processing uses at least a multilayer perceptron neural network. 
     
     
       11. A method comprising:
 receiving input text, wherein the input text comprises text data, a natural language question, and an answer candidate for the natural language question based on the text data; 
 converting the input text into a word-by-word representation based at least on a word-by-word sentential match between a combination of the natural language question and the answer candidate and the text data; 
 converting the input text into a semantic representation of the input text based at least on a semantic match between the combination of the natural language question and the answer candidate and the text data; 
 concurrently processing:
 generating a first result based on analyzing the word-by-word representation using one or more word-by-word processes, wherein each word-by-word process generates a first matching score indicating a degree of the word-by-word sentential match; and 
 generating a second result based on analyzing the semantic representation using semantic processes, wherein each semantic process generates a second matching score indicating a degree of the semantic match; 
 
 combining the first result and the second result 
 determining, based on the combined first and second results, a top result, wherein the top result includes the combination of the natural language question and the answer candidate corresponding to the highest score among a set of matching scores including the first matching score and the second matching score; and 
 providing the top result. 
 
     
     
       12. The method of  claim 11 ,
 wherein each word-by-word processes use a multilayer perceptron neural network including the natural language question and the answer candidate, 
 wherein the semantic processes use a multilayer perceptron plus sum neural network based on semantics of the input text with a summation using a weight associated with a word, and 
 wherein the first result and the second result are distinct. 
 
     
     
       13. The method of  claim 12 , wherein the semantic processes include comparing a hypothesis to sentences in the text data. 
     
     
       14. The method of  claim 13 , wherein the hypothesis includes a combination of at least a portion of the natural language question with at least a portion of the answer candidate. 
     
     
       15. The method of  claim 14 , wherein the hypothesis is compared to the text data using cosine similarity. 
     
     
       16. The method of  claim 11 , wherein the one or more word-by-word process comprises at least one of:
 a sentential process; 
 a sliding window sequential process; and 
 a dependency sliding window dependency process. 
 
     
     
       17. The method of  claim 16 , wherein the sliding window sequential process scans over words of the text data as one continuous sequence. 
     
     
       18. The method of  claim 16 , wherein the dependency sliding window dependency process comprises:
 constructing a dependency graph for a sentence in the text data; 
 reordering words in the sentence based at least in part on the dependency graph to generate a reordered sentence; and 
 scans over words of the reordered sentence. 
 
     
     
       19. A computer storage medium comprising computer executable instructions that, when executed by at least one processor, executes a method comprising:
 receiving input text, wherein the input text comprises text data, a natural language question, and an answer candidate for the natural language question based on the text data; 
 converting the input text into a word-by-word representation based at least on a word-by-word sentential match between a combination of the natural language question and the answer candidate and the text data; 
 converting the input text into a semantic representation of the input text based at least on a semantic match between the combination of the natural language question and the answer candidate and the text data; 
 concurrently processing:
 generating a first result based on analyzing the word-by-word representation using one or more word-by-word processes, wherein each word-by-word process generates a first matching score indicating a degree of the word-by-word sentential match; and 
 generating a second result based on analyzing the semantic representation using semantic processes, wherein each semantic process generates a second matching score indicating a degree of the semantic match; 
 
 combining the first result and the second result; 
 determining, based on the combined first and second results, a top result, wherein the top result includes the combination of the natural language question and the answer candidate corresponding to the highest score among a set of matching scores including the first matching score and the second matching score; and 
 providing the top result. 
 
     
     
       20. The computer storage medium of  claim 19 , wherein each word-by-word processes use a multilayer perceptron neural network including the natural language question and the answer candidate,
 wherein the semantic processes use a multilayer perceptron plus sum neural network based on semantics of the input text based on semantics of the input text with a summation using a weight associated with a word, and 
 wherein the first result and the second result are distinct.