Classification based on a knowledge model combined with a machine learning based model

Abstract

A system makes predictions using a machine learning model combined with a knowledge model. The system provides input data to a knowledge model and a machine learning based model. The machine learning based model is trained to make predictions based on input data. The system provides the outputs of the machine learning based model and the knowledge model to an ensemble model configured to combine results of the knowledge model and the machine learning based model. The system can be used for several applications. For example, the system may classify an input text based on a hierarchy of categories. The system may perform fault detection in time series data by identifying an anomaly data point and predicting whether the anomaly data point is a fault.

What is claimed is: 
     
         1 . A computer-implemented method for text classification comprising: 
 receiving an input text for classification based on a hierarchy of categories;   providing the input text to a knowledge model, wherein the knowledge model is a rule-based model comprising rules for classifies text;   providing the input text to a machine learning based model trained for classifying text;   executing the knowledge model to generate a first output representing a first category for the input text;   executing the machine learning based model to generate a second output representing a second category for the input text;   providing the first output and the second output to an ensemble model configured to combine results of the knowledge model and the machine learning based model;   executing the ensemble model to determine a category for the input text based on the first category and the second category; and   sending the category for the input text determined by the ensemble model to a client device.   
     
     
         2 . The computer-implemented method of  claim 1 , further comprising:
 receiving a new set of text inputs for classifying, wherein the machine learning based model has low accuracy of classification for text inputs from the new set of text inputs; and   adding one or more rules to the knowledge model for classifying documents of the new set of text inputs, wherein the ensemble model determines the category of an input text from the new set of text inputs as the second category responsive to determining that an accuracy of classification of the machine learning based model for the input text from the new set of text inputs is below a threshold value.   
     
     
         3 . The computer-implemented method of  claim 2 , further comprising:
 using the input text from the new set of text inputs and the category determined for the input text by the ensemble model as training data for training the machine learning based model.   
     
     
         4 . The computer-implemented method of  claim 2 , further comprising:
 generating synthetic data based on the input text from the new set of text inputs and the category determined for the input text by the ensemble model as additional training data for the machine learning based model.   
     
     
         5 . The computer-implemented method of  claim 1 , wherein determining the category of the input text by the ensemble model comprises:
 receiving a first measure of accuracy of the first output generated by the knowledge model;   receiving a second measure of accuracy of the second output generated by the machine learning based model; and   determining the category for the input text based on the first output and the second output based on at least one of the first measure of accuracy or the second measure of accuracy.   
     
     
         6 . The computer-implemented method of  claim 5 , wherein the ensemble model determines that the category of the input text is the first category if a comparison of the first measure of accuracy and the second measure of accuracy indicates that the knowledge model has higher accuracy compared to the machine learning based model. 
     
     
         7 . The computer-implemented method of  claim 1 , wherein the input text represents articles retrieved from a website. 
     
     
         8 . A non-transitory computer readable storage medium storing instructions that when executed by one or more computer processors, cause the one or more computer processors to perform steps comprising:
 receiving an input text for classification based on a hierarchy of categories;   providing the input text to a knowledge model, wherein the knowledge model is a rule-based model comprising rules for classifies text;   providing the input text to a machine learning based model trained for classifying text;   executing the knowledge model to generate a first output representing a first category for the input text;   executing the machine learning based model to generate a second output representing a second category for the input text;   providing the first output and the second output to an ensemble model configured to combine results of the knowledge model and the machine learning based model;   executing the ensemble model to determine a category for the input text based on the first category and the second category; and   sending the category for the input text determined by the ensemble model to a client device.   
     
     
         9 . The non-transitory computer readable storage medium of  claim 8 , wherein the instructions further cause the one or more computer processors to perform steps comprising:
 receiving a new set of text inputs for classifying, wherein the machine learning based model has low accuracy of classification for text inputs from the new set of text inputs; and   adding one or more rules to the knowledge model for classifying documents of the new set of text inputs, wherein the ensemble model determines the category of an input text from the new set of text inputs as the second category responsive to determining that an accuracy of classification of the machine learning based model for the input text from the new set of text inputs is below a threshold value.   
     
     
         10 . The non-transitory computer readable storage medium of  claim 9 , wherein the instructions further cause the one or more computer processors to perform steps comprising:
 using the input text from the new set of text inputs and the category determined for the input text by the ensemble model as training data for training the machine learning based model.   
     
     
         11 . The non-transitory computer readable storage medium of  claim 9 , further comprising:
 generating synthetic data based on the input text from the new set of text inputs and the category determined for the input text by the ensemble model as additional training data for the machine learning based model.   
     
     
         12 . The non-transitory computer readable storage medium of  claim 8 , wherein instructions for determining the category of the input text by the ensemble model cause the one or more computer processors to perform steps comprising:
 receiving a first measure of accuracy of the first output generated by the knowledge model;   receiving a second measure of accuracy of the second output generated by the machine learning based model; and   determining the category for the input text based on the first output and the second output based on at least one of the first measure of accuracy or the second measure of accuracy.   
     
     
         13 . The non-transitory computer readable storage medium of  claim 12 , wherein the ensemble model determines that the category of the input text is the first category if a comparison of the first measure of accuracy and the second measure of accuracy indicates that the knowledge model has higher accuracy compared to the machine learning based model. 
     
     
         14 . The non-transitory computer readable storage medium of  claim 8 , wherein the input text represents articles retrieved from a website. 
     
     
         15 . A computer system comprising:
 one or more computer processors; and   a non-transitory computer readable storage medium storing instructions that when executed by the one or more computer processors, cause the one or more computer processors to perform steps comprising: 
 receiving an input text for classification based on a hierarchy of categories; 
 providing the input text to a knowledge model, wherein the knowledge model is a rule-based model comprising rules for classifies text; 
 providing the input text to a machine learning based model trained for classifying text; 
 executing the knowledge model to generate a first output representing a first category for the input text; 
 executing the machine learning based model to generate a second output representing a second category for the input text; 
 providing the first output and the second output to an ensemble model configured to combine results of the knowledge model and the machine learning based model; 
 executing the ensemble model to determine a category for the input text based on the first category and the second category; and 
 sending the category for the input text determined by the ensemble model to a client device. 
   
     
     
         16 . The computer system of  claim 15 , wherein the instructions further cause the one or more computer processors to perform steps comprising:
 receiving a new set of text inputs for classifying, wherein the machine learning based model has low accuracy of classification for text inputs from the new set of text inputs; and   adding one or more rules to the knowledge model for classifying documents of the new set of text inputs, wherein the ensemble model determines the category of an input text from the new set of text inputs as the second category responsive to determining that an accuracy of classification of the machine learning based model for the input text from the new set of text inputs is below a threshold value.   
     
     
         17 . The computer system of  claim 16 , wherein the instructions further cause the one or more computer processors to perform steps comprising:
 using the input text from the new set of text inputs and the category determined for the input text by the ensemble model as training data for training the machine learning based model.   
     
     
         18 . The computer system of  claim 16 , wherein the instructions further cause the one or more computer processors to perform steps comprising:
 generating synthetic data based on the input text from the new set of text inputs and the category determined for the input text by the ensemble model as additional training data for the machine learning based model.   
     
     
         19 . The computer system of  claim 15 , wherein determining the category of the input text by the ensemble model causes the one or more computer processors to perform steps comprising:
 receiving a first measure of accuracy of the first output generated by the knowledge model;   receiving a second measure of accuracy of the second output generated by the machine learning based model; and   determining the category for the input text based on the first output and the second output based on at least one of the first measure of accuracy or the second measure of accuracy.   
     
     
         20 . The computer system of  claim 19 , wherein the ensemble model determines that the category of the input text is the first category if a comparison of the first measure of accuracy and the second measure of accuracy indicates that the knowledge model has higher accuracy compared to the machine learning based model.