Neural tagger with deep multi-level model

Abstract

Embodiments of the described technologies are capable of reading a text sequence that include at least one word; extracting model input data from the text sequence, where the model input data includes, for each word of the text sequence, segment data and non-segment data; using a first machine learning model and at least one second machine learning model, generating, for each word of the text sequence, a multi-level feature set; outputting, by a third machine learning model, in response to input to the third machine learning model of the multi-level feature set, a tagged version of the text sequence; executing a search based at least in part on the tagged version of the text sequence.

What is claimed is: 
     
       1. A method comprising, by an information retrieval system:
 reading a text sequence that comprises at least one word; 
 extracting model input data from the text sequence, wherein the model input data comprises, for each word of the text sequence, segment data and non-segment data, wherein the segment data is generated by determining a text sequence length threshold and a segment length threshold different from the text sequence length threshold, and, if a length of the text sequence is less than or equal to the text sequence length threshold, for each word of the text sequence, generating a plurality of segments of the text sequence that each are less than or equal to the segment length threshold; 
 using a first machine learning model generating, for each word of the text sequence, based on the model input data, a first feature subset of a multi-level feature set; 
 using a second machine learning model generating, for each word of the text sequence, based on the model input data, a second feature subset of the multi-level feature set; 
 outputting, by a third machine learning model, in response to input to the third machine learning model of the multi-level feature set, a tagged version of the text sequence; 
 executing a search based at least in part on the tagged version of the text sequence. 
 
     
     
       2. The method of  claim 1 , wherein the non-segment data comprises, for each word of the text sequence, at least one of (i) character-level data extracted from the word, (ii) word-level data extracted from the word, or (iii) any combination of (i) and (ii). 
     
     
       3. The method of  claim 1 , wherein the multi-level feature set comprises, for each word of the text sequence, the first feature subset comprising at least one segment-level feature subset and at least the second feature subset comprising one non-segment-level feature subset. 
     
     
       4. The method of  claim 3 , wherein a segment-level feature subset of the multi-level feature set is output by the first machine learning model in response to input to the first machine learning model, of the segment data. 
     
     
       5. The method of  claim 3 , wherein a non-segment-level feature subset of the multi-level feature set is output by the second machine learning model in response to input, to the second machine learning model, of at least one of the segment data and the non-segment data. 
     
     
       6. The method of  claim 1 , wherein the first machine learning model is trained on segment training data generated by determining a text sequence length threshold and a segment length threshold, and, for segment training data having a length less than or equal to the text sequence length threshold, generating a plurality of segments of the segment training data that each are less than or equal to the segment length threshold. 
     
     
       7. The method of  claim 1 , wherein the third machine learning model is generating the tagged version of the text sequence by grouping words of the text sequence according to matching tags. 
     
     
       8. A method comprising, by a device operating a software-based service:
 determining a text sequence obtained from a software application; 
 the text sequence comprising at least one word; 
 extracting model input data from the text sequence, wherein the model input data comprises, for each word of the text sequence, segment data and non-segment data, wherein the segment data is generated by determining a text sequence length threshold and a segment length threshold different from the text sequence length threshold, and, if a length of the text sequence is less than or equal to the text sequence length threshold, for each word of the text sequence, generating a plurality of segments of the text sequence that each are less than or equal to the segment length threshold; 
 by a first machine learning model, generating, in response to input of the model input data to the first machine learning model, for each word of the text sequence, a first feature subset of a multi-level feature subset; 
 by a second machine learning model, generating, in response to input of the model input data to the second machine learning model, for each word of the text sequence, a second feature set of the multi-level feature set; 
 outputting, by a third machine learning model, in response to input of the multi-level feature set to the third machine learning model, a tagged version of the text sequence for processing by the software application. 
 
     
     
       9. The method of  claim 8 , wherein the multi-level feature set comprises tag prediction data; the tag prediction data output by a fourth machine learning model in response to input to the fourth machine learning model, of output of at least one of (i) a segment-level deep neural network model, (ii) a word-level deep neural network model, (iii) a character-level deep neural network model, (iv) any combination of (i), (ii), (iii), (iv). 
     
     
       10. The method of  claim 8 , wherein the multi-level feature set comprises tag prediction data; the tag prediction data output by a statistical model in response to input, to the statistical model of at least one of (i) the segment data, (ii) word-level data of the non-segment data, (iii) both (i) and (ii). 
     
     
       11. The method of  claim 8 , wherein the multi-level feature set comprises first tag prediction data and second tag prediction data; the first tag prediction data output by a fourth machine learning model in response to input, to the fourth machine learning model, of output of at least one of (i) a segment-level deep neural network model, (ii) a word-level deep neural network model, (iii) a character-level deep neural network model, (iv) any combination of (i), (ii), (iii); the second tag prediction data output by a statistical model in response to input, to the statistical model of at least one of (v) the segment data, (vi) word-level data of the non-segment data, (vii) both (v) and (vi). 
     
     
       12. The method of  claim 8 , wherein the multi-level feature set comprises tag prediction data; the tag prediction data output by a fourth machine learning model in response to input, to the fourth machine learning model, of output of at least two of (i) a segment-level deep neural network model, (ii) a word-level deep neural network model, (iii) a character-level deep neural network model, (iv) any combination of (i), (ii), (iii). 
     
     
       13. The method of  claim 8 , wherein the multi-level feature set comprises tag prediction data; the tag prediction data output by a statistical model in response to input, to the statistical model of the segment data and word-level data of the non-segment data. 
     
     
       14. The method of  claim 8 , wherein the multi-level feature set comprises first tag prediction data and second tag prediction data; the first tag prediction data output by a fourth machine learning model in response to input, to the fourth machine learning model, of output of a segment-level deep neural network model and output of a word-level deep neural network model and output of a character-level deep neural network model; the second tag prediction data output by a statistical model in response to input, to the statistical model of the segment data and word-level data of the non-segment data. 
     
     
       15. A system, comprising:
 at least one processor; 
 at least one computer memory operably coupled to the at least one processor; 
 the at least one computer memory configured according to a multi-level model; 
 the multi-level model comprising a first machine learning model to generate, for each word of a text sequence, a segment-level feature set of a multi-level feature set and a model to generate, for each word of the text sequence, a non-segment level feature set of the multi-level feature set; 
 the first machine learning model comprising a deep neural network trained on segment training data; 
 the segment training data generated by determining a text sequence length threshold and a segment length threshold different from the text sequence length threshold, and, for text sequence training data having a length less than or equal to the text sequence length threshold, generating a plurality of segments of the text sequence training data that each are less than or equal to the segment length threshold. 
 
     
     
       16. The system of  claim 15 , wherein the model of the multi-level model comprises a statistical model configured to be communicatively coupled to a pre-determined lexicon. 
     
     
       17. The system of  claim 16 , an input of the statistical model capable of being communicatively coupled to an output of a segmentation mechanism; the segmentation mechanism capable of generating segment data from the text sequence by determining a text sequence length threshold and a segment length threshold, and, if the text sequence has a length that is less than or equal to the text sequence length threshold, generating a plurality of segments of the text sequence that each are less than or equal to the segment length threshold. 
     
     
       18. The system of  claim 16 , wherein the model of the multi-level model comprises at least one deep neural network trained on non-segment data; the non-segment data comprising at least one of word-level data extracted from the text sequence training data and character-level data extracted from the text sequence training data. 
     
     
       19. The system of  claim 18 , wherein the multi-level model further comprises a third machine learning model; an input of the third machine learning model configured to be communicatively coupled to an output of the first machine learning model and an output of the at least one deep neural network trained on non-segment data.