Constituent Centric Architecture for Reading Comprehension

Abstract

A constituent-centric neural architecture for reading comprehension is disclosed. One embodiment provides a method that performs reading comprehension comprising encoding individual constituents from a text passage using a chain of trees long short-term encoding, encodes question related to the text passage using a tree long short-term memory encoding, generates a question-aware representation for each constituent in the passage using a tree-guided attention mechanism, generates a plurality of candidate answers from the question-aware representation using hierarchical relations among constituents, and predicts an answer to the question in relation to the text passage using a feed-forward network. Other embodiments are disclosed herein.

1 . A method for performing reading comprehension (RC), comprising:
 encoding individual constituents from a text passage using a chain of trees long short-term encoding;   encoding a question related to the text passage using a tree long short-term memory encoding;   generating a question-aware representation for each constituent in the passage using a tree-guided attention mechanism;   generating a plurality of candidate answers from the question-aware representation using hierarchical relations among constituents; and   predicting an answer to the question in relation to the text passage using a feed-forward network.   
     
     
         2 . The method of  claim 1 , wherein encoding individual constituents from the text passage using a chain of trees long short-term encoding further comprises:
 building a bi-directional tree long short-term memory encoding for each sentence; and   gluing together the bi-directional tree long short-term memory encoding with a bi-directional chain long short-term memory.   
     
     
         3 . The method of  claim 1 , wherein the chain of trees long short-term memory encoding further comprises:
 computing hidden states for each sentence in the passage using bottom-up tree long short-term memory encodings and feeding the hidden states for each sentence into a root node of the chain of trees long short-term memory;   computing forward and backward states in the chain of trees long short-term memory and feeding the forward and backward states into a root of the top-down tree long short-term memory; and   computing top down hidden states capturing semantics of the passage.   
     
     
         4 . The method of  claim 1 , wherein encoding a question related to the text passage using a tree long short-term memory encoding further comprises encoding constituents using a bi-directional long short-term memory and a top-down long short-term memory. 
     
     
         5 . The method of  claim 1 , wherein the tree-guided attention mechanism further comprises:
 measuring similarity between a constituent in the text passage and a constituent in the question using a constituent-level attention score computation;   generating locally normalized attention scores using a tree-guided local normalization; and   generating a tree-guided attentional summarization.   
     
     
         6 . The method of  claim 1 , wherein generating a plurality of candidate answers from the question-aware representation using hierarchical relations among constituents further comprises:
 expanding each constituent by appending adjacent words;   performing reduction on the expanded constituents by removing overlap from duplicated expansions; and   encoding candidate answers using a bi-directional chain long short-term memory mechanism.   
     
     
         7 . The method of  claim 1 , wherein predicting an answer to the question in relation to the text passage using a feed-forward network further comprises receiving a feature vector and generating a confidence score for each candidate answer. 
     
     
         8 . The method of  claim 7 , further comprising:
 normalizing confidence scores for the candidate answers into a probabilistic simplex using softmax; and   defining a cross entropy loss of the normalized confidence scores.   
     
     
         9 . A reading comprehension (RC) system with memory and a processor, the reading comprehension system comprising:
 a parsing module; and   a Constituent-Centric Neural Network (CCNN) module stored in memory and comprising an encoding module to encode constituents in a text passage and one or more text questions, a tree-guided attention module to learn question-aware representations, a candidate answer generation module, the candidate answer generation module to produce candidate answers based on a constituent expansion, and an answer prediction module to select the best answer from the candidate answers using a feed-forward network.   
     
     
         10 . The system of  claim 9 , wherein the encoding module encodes individual constituents from the text passage using a chain of trees long short-term encoding by building a bi-directional tree long short-term memory encoding for each sentence and gluing together the bi-directional tree long short-term memory encoding with a bi-directional chain long short-term memory. 
     
     
         11 . The system of  claim 9 , wherein the tree-guided attention module is a chain of trees long short-term memory configured to compute hidden states for each sentence in the passage using bottom-up tree long short-term memory encodings and feed the hidden states for each sentence into a root node of the chain of trees long short-term memory;
 compute forward and backward states in the chain of trees long short-term memory and feeding the forward and backward states into a root of the top-down tree long short-term memory; and   compute top down hidden states capturing semantics of the passage.   
     
     
         12 . The system of  claim 9 , wherein the encoding module encodes a question related to the text passage using a tree long short-term memory encoding. 
     
     
         13 . The system of  claim 12 , the encoding module to encode constituents using a bi-directional long short-term memory and a top-down long short-term memory 
     
     
         14 . The system of  claim 9 , wherein the tree-guided attention module is to measure similarity between a constituent in the text passage and a constituent in the question using a constituent-level attention score computation;
 generate locally normalized attention scores using a tree-guided local normalization; and   generate a tree-guided attentional summarization.   
     
     
         15 . The system of  claim 9 , wherein the candidate answer module is configured to expand each constituent by appending adjacent words;
 perform reduction on the expanded constituents by removing overlap from duplicated expansions; and   encode candidate answers using a bi-directional chain long short-term memory mechanism.   
     
     
         16 . The system of  claim 1 , wherein to predict an answer to the question in relation to the text passage uses a feed-forward network further configured to receive a feature vector and generate a confidence score for each candidate answer. 
     
     
         17 . The system of  claim 16 , wherein the answer prediction module is configured to normalize confidence scores for the candidate answers into a probabilistic simplex using softmax; and
 define a cross entropy loss of the normalized confidence scores.   
     
     
         18 . A method for performing reading comprehension (RC), comprising:
 parsing a text passage and one or more text questions into constituents;   encoding said constituents in an encoding sub-module and sending encoded constituents to a tree guided attention sub-module to learn question-aware representations;   receiving said question-aware representations in a candidate answer generation sub-module to generate candidate answers; and   selecting the best answer from said candidate answers in an answer prediction sub-module.