Feature crossing for machine learning

Abstract

Embodiments of the present disclosure relate to feature crossing for machine learning. According to example embodiments of the present disclosure, a method comprises determining a semantic correlation relationship between a plurality of feature categories, the semantic correlation relationship indicating respective degrees of semantic correlation between respective pairs of feature categories among the plurality of feature categories; obtaining at least two features classified in at least two of the plurality of feature categories for machine learning; and performing feature crossing on the at least two features based on the semantic correlation relationship.

What is claimed is: 
     
       1. A method comprising:
 determining a semantic correlation relationship between a plurality of feature categories, the semantic correlation relationship indicating respective degrees of semantic correlation between respective pairs of feature categories among the plurality of feature categories, wherein the determining the semantic correlation relationship comprises:
 identifying a set of named entities from a corpus, the corpus comprising a plurality of text sequences, 
 classifying the set of named entities into a plurality of entity clusters, each of the plurality of entity clusters being associated with one of the plurality of feature categories, and 
 determining the respective degrees of semantic correlation among the plurality of feature categories based on relative positioning of the plurality of entity clusters within the plurality of text sequences; 
 
 obtaining at least two features classified in at least two of the plurality of feature categories for machine learning; and 
 performing feature crossing on the at least two features based on the semantic correlation relationship, wherein the performing the feature crossing comprises: 
 determining a target degree of semantic correlation between the at least two feature categories based on the semantic correlation relationship, 
 in accordance with a determination that the target degree of semantic correlation exceeds a threshold degree of semantic correlation, applying a feature crossing operation on the at least two features to generate a crossed feature, and 
 in accordance with a determination that the target degree of semantic correlation is below the threshold degree of semantic correlation, ceasing to apply the feature crossing operation on the at least two features. 
 
     
     
       2. The method of  claim 1 , wherein determining the semantic correlation relationship comprises:
 transforming respective names of the plurality of feature categories into a plurality of embedding representations using a language model; and 
 determining the respective degrees of semantic correlation based on respective similarities between the plurality of embedding representations. 
 
     
     
       3. The method of  claim 1 , wherein identifying the set of named entities comprises:
 obtaining a named entity recognition model, the named entity recognition model being trained with a plurality of sample text sequences and first labeling information indicating named entities occurred in the plurality of sample text sequences; and 
 identifying the set of named entities using the named entity recognition model. 
 
     
     
       4. The method of  claim 1 , wherein classifying the set of named entities comprises:
 transforming the set of named entities into a set of embedding representations using a language model; and 
 clustering the set of named entities based on respective similarities between the set of embedding representations. 
 
     
     
       5. The method of  claim 1 , wherein classifying the set of named entities comprises:
 obtaining a classification model, the classification model being trained with a plurality of sample named entities and second labeling information, the second labeling information indicating a classification result of the plurality of sample named entities into the plurality of entity clusters; and 
 classifying the set of named entities using the classification model. 
 
     
     
       6. The method of  claim 1 , wherein determining the respective degrees of semantic correlation based on the relative positioning of the plurality of named entities within the plurality of text sequences comprises:
 for a first entity cluster and a second entity cluster of the plurality of clusters,
 determining a frequency of co-occurrence of a first named entity classified in the first entity cluster in relation to a second named entity classified in the second entity cluster within same text sequences of the plurality of text sequences; and 
 determining, based on the frequency of co-occurrence, a degree of semantic correlation between a first feature category associated with the first entity cluster and a second feature category associated with the second entity cluster. 
 
 
     
     
       7. The method of  claim 1 , wherein determining the semantic correlation relationship comprises:
 generating a relational graph to indicate the semantic correlation relationship, the relational graph comprising a plurality of vertexes representing the plurality of feature categories, and a plurality of edges connecting the plurality of vertexes, each of the plurality of edges indicating a degree of semantic correlation between two feature categories represented by two vertexes connected by the edge. 
 
     
     
       8. The method of  claim 1 , wherein the semantic correlation relationship indicates respective degrees of semantic correlation between respective pairs of feature categories among the plurality of feature categories; and
 wherein determining the target degree of semantic correlation comprises:
 in accordance with a determination that more than two features classified in more than two feature categories are obtained, 
 determining, from the semantic correlation relationship, at least two degrees of semantic correlation between at least two pairs of feature categories among the more than two feature categories, and 
 determining the target degree of semantic correlation by combining the at least two degrees of semantic correlation. 
 
 
     
     
       9. A system, comprising:
 at least one processor; and 
 at least one memory communicatively coupled to the at least one processor and comprising computer-readable instructions that upon execution by the at least one processor cause the at least one processor to perform acts comprising: 
 determining a semantic correlation relationship between a plurality of feature categories, the semantic correlation relationship indicating respective degrees of semantic correlation between respective pairs of feature categories among the plurality of feature categories, wherein the determining the semantic correlation relationship comprises:
 identifying a set of named entities from a corpus, the corpus comprising a plurality of text sequences, 
 classifying the set of named entities into a plurality of entity clusters, each of the plurality of entity clusters being associated with one of the plurality of feature categories, and 
 determining the respective degrees of semantic correlation among the plurality of feature categories based on relative positioning of the plurality of entity clusters within the plurality of text sequences; 
 
 obtaining at least two features classified in at least two of the plurality of feature categories for machine learning; and 
 performing feature crossing on the at least two features based on the semantic correlation relationship, wherein the performing the feature crossing comprises: 
 determining a target degree of semantic correlation between the at least two feature categories based on the semantic correlation relationship, 
 in accordance with a determination that the target degree of semantic correlation exceeds a threshold degree of semantic correlation, applying a feature crossing operation on the at least two features to generate a crossed feature, and 
 in accordance with a determination that the target degree of semantic correlation is below the threshold degree of semantic correlation, ceasing to apply the feature crossing operation on the at least two features. 
 
     
     
       10. The system of  claim 9 , wherein determining the semantic correlation relationship comprises:
 transforming respective names of the plurality of feature categories into a plurality of embedding representations using a language model; and 
 determining the respective degrees of semantic correlation based on respective similarities between the plurality of embedding representations. 
 
     
     
       11. The system of  claim 9 , wherein identifying the set of named entities comprises:
 obtaining a named entity recognition model, the named entity recognition model being trained with a plurality of sample text sequences and first labeling information indicating named entities occurred in the plurality of sample text sequences; and 
 identifying the set of named entities using the named entity recognition model. 
 
     
     
       12. The system of  claim 9 , wherein classifying the set of named entities comprises:
 transforming the set of named entities into a set of embedding representations using a language model; and 
 clustering the set of named entities based on respective similarities between the set of embedding representations. 
 
     
     
       13. The system of  claim 9 , wherein classifying the set of named entities comprises:
 obtaining a classification model, the classification model being trained with a plurality of sample named entities and second labeling information, the second labeling information indicating a classification result of the plurality of sample named entities into the plurality of entity clusters; and 
 classifying the set of named entities using the classification model. 
 
     
     
       14. The system of  claim 9 , wherein determining the respective degrees of semantic correlation based on the relative positioning of the plurality of named entities within the plurality of text sequences comprises:
 for a first entity cluster and a second entity cluster of the plurality of clusters,
 determining a frequency of co-occurrence of a first named entity classified in the first entity cluster in relation to a second named entity classified in the second entity cluster within same text sequences of the plurality of text sequences; and 
 determining, based on the frequency of co-occurrence, a degree of semantic correlation between a first feature category associated with the first entity cluster and a second feature category associated with the second entity cluster. 
 
 
     
     
       15. The system of  claim 9  wherein the semantic correlation relationship indicates respective degrees of semantic correlation between respective pairs of feature categories among the plurality of feature categories; and
 wherein determining the target degree of semantic correlation comprises:
 in accordance with a determination that more than two features classified in more than two feature categories are obtained, 
 determining, from the semantic correlation relationship, at least two degrees of semantic correlation between at least two pairs of feature categories among the more than two feature categories, and 
 determining the target degree of semantic correlation by combining the at least two degrees of semantic correlation. 
 
 
     
     
       16. A non-transitory computer-readable storage medium, storing computer-readable instructions that upon execution by a computing device cause the computing device to perform acts comprising:
 determining a semantic correlation relationship between a plurality of feature categories, the semantic correlation relationship indicating respective degrees of semantic correlation between respective pairs of feature categories among the plurality of feature categories, wherein the determining the semantic correlation relationship comprises:
 identifying a set of named entities from a corpus, the corpus comprising a plurality of text sequences, 
 classifying the set of named entities into a plurality of entity clusters, each of the plurality of entity clusters being associated with one of the plurality of feature categories, and 
 determining the respective degrees of semantic correlation among the plurality of feature categories based on relative positioning of the plurality of entity clusters within the plurality of text sequences; 
 
 obtaining at least two features classified in at least two of the plurality of feature categories for machine learning; and 
 performing feature crossing on the at least two features based on the semantic correlation relationship, wherein the performing the feature crossing comprises: 
 determining a target degree of semantic correlation between the at least two feature categories based on the semantic correlation relationship, 
 in accordance with a determination that the target degree of semantic correlation exceeds a threshold degree of semantic correlation, applying a feature crossing operation on the at least two features to generate a crossed feature, and 
 in accordance with a determination that the target degree of semantic correlation is below the threshold degree of semantic correlation, ceasing to apply the feature crossing operation on the at least two features. 
 
     
     
       17. The non-transitory computer-readable storage medium of  claim 16 , wherein the determining the semantic correlation relationship comprises:
 transforming respective names of the plurality of feature categories into a plurality of embedding representations using a language model; and 
 determining the respective degrees of semantic correlation based on respective similarities between the plurality of embedding representations. 
 
     
     
       18. The non-transitory computer-readable storage medium of  claim 16 , wherein the identifying the set of named entities comprises:
 obtaining a named entity recognition model, the named entity recognition model being trained with a plurality of sample text sequences and first labeling information indicating named entities occurred in the plurality of sample text sequences; and 
 identifying the set of named entities using the named entity recognition model. 
 
     
     
       19. The non-transitory computer-readable storage medium of  claim 16 , wherein the classifying the set of named entities comprises:
 transforming the set of named entities into a set of embedding representations using a language model; and 
 clustering the set of named entities based on respective similarities between the set of embedding representations. 
 
     
     
       20. The non-transitory computer-readable storage medium of  claim 16 , wherein the determining the semantic correlation relationship comprises:
 generating a relational graph to indicate the semantic correlation relationship, the relational graph comprising a plurality of vertexes representing the plurality of feature categories, and a plurality of edges connecting the plurality of vertexes, each of the plurality of edges indicating a degree of semantic correlation between two feature categories represented by two vertexes connected by the edge.