Method of generating features optimal to a dataset and classifier

Abstract

A method of generating features optimal to a particular dataset and classifier is disclosed. A dataset of messages is inputted and a classifier is selected. An algebra of features is encoded. Computable features that are capable of describing the dataset from the algebra of features are selected. Irredundant features that are optimal for the classifier and the dataset are selected.

We claim: 
     
         1 . A method of generating features optimal to a particular dataset and classifier, comprising:
 a. inputting a dataset of messages;   b. inputting a classifier;   c. encoding an algebra of features;   d. selecting computable features capable of describing the dataset from the algebra of features; and   e. selecting an optimal feature that is optimal for the classifier and the dataset.   
     
     
         2 . The method of  claim 1  further comprising randomly selecting a first subset of the computable features and inputting them into the classifier. 
     
     
         3 . The method of  claim 2  further comprising determining accuracy of the classifier using a validation set. 
     
     
         4 . The method of  claim 3  further comprising randomly selecting a second subset of features from the first subset of computable features. 
     
     
         5 . The method of  claim 4  further comprising mutating the second subset of features. 
     
     
         6 . The method of  claim 5  further comprising randomly selecting pairs of features from the first subset of computable features. 
     
     
         7 . The method of  claim 6  further comprising applying crossing to each selected pair of features from the first subset of computable features. 
     
     
         8 . The method of  claim 7  further comprising combining the first subset of computable features, the mutated features, and the crossed features to generate a third subset of features. 
     
     
         9 . The method of  claim 8  further comprising selecting a fourth subset of features from the algebra of features and adding the fourth subset of features to the third subset of features. 
     
     
         10 . The method of  claim 9  further comprising removing duplicate features from the third subset of features. 
     
     
         11 . The method of  claim 10  further comprising inputting the features from the third subset into the classifier and computing accuracy of the classifier. 
     
     
         12 . The method of  claim 11  further comprising ranking the features of the third subset by their corresponding classifier accuracy. 
     
     
         13 . The method of  claim 12  further comprising breaking ties, if any, in the ranking based on a total order. 
     
     
         14 . The method of  claim 13  further comprising maintaining only fittest features from the third subset to form a fifth subset of the same size as the first subset, and replacing the first subset with the fifth subset. 
     
     
         15 . The method of  claim 14  further comprising repeating the mutating, the crossing, the removing, the ranking, the selecting, and the maintaining until accuracy of a most fit feature converges, wherein the most fit feature is cached.