Graph-based transfer learning

Abstract

Transfer learning is the task of leveraging the information from labeled examples in some domains to predict the labels for examples in another domain. It finds abundant practical applications, such as sentiment prediction, image classification and network intrusion detection. A graph-based transfer learning framework propagates label information from a source domain to a target domain via the example-feature-example tripartite graph, and puts more emphasis on the labeled examples from the target domain via the example-example bipartite graph. An iterative algorithm renders the framework scalable to large-scale applications. The framework propagates the label information to both features irrelevant to the source domain and unlabeled examples in the target domain via common features in a principled way.

1 . A computer method, comprising carrying out operations on a computer, the operations comprising:
 maintaining machine readable embodiments on a medium of first and second graphs,
 the first graph comprising
 a first plurality of nodes corresponding to labeled and unlabeled examples from source and target domains; 
 a second plurality of nodes corresponding to features; and 
 a first plurality of edges connecting
 the nodes corresponding to the features 
 to the nodes corresponding to the examples according to whether the features appear in the examples or not; 
 
 
 the second graph comprising
 the first plurality of nodes corresponding to the examples; and 
 a second plurality of edges connecting the examples, the edges being associated with indications that indicate whether connected examples are in a same domain or not; 
 
   deriving labels for at least one target domain based on the first and second graphs; and   presenting an embodiment of the labels as a result.   
     
     
         2 . The method of  claim 1 , wherein the labels in the source domain are related to a first field of application, while the labels in the target domain are related to a second field of application. 
     
     
         3 . The method of  claim 1 , wherein initially the labels represent sentiments expressed by users with respect to examples, while derived labels represent anticipated sentiments with respect to unlabelled examples. 
     
     
         4 . The method of  claim 1 , wherein initially the labels represent known document classifications with respect to examples, while derived labels represent anticipated document classifications with respect to unlabelled examples. 
     
     
         5 . The method of  claim 1 , wherein initially the labels represent known intrusion detection results with respect to examples, while derived labels represent anticipated intrusion detection results with respect to unlabelled examples. 
     
     
         6 . The method of  claim 1 , wherein deriving takes into account labels from both the target and source domains, but weights the labels from the target domain more heavily in determining derived labels for unlabelled examples in the target domain. 
     
     
         7 . The method of  claim 1 , wherein the examples are machine readable embodiments—in a medium—of documents; and the features are machine embodiments—in a medium—of words within the documents. 
     
     
         8 . The method of  claim 1 , wherein deriving comprises imposing at least one smoothness constraint on the graphs. 
     
     
         9 . The method of  claim 1 , wherein deriving comprises imposing at least one label consistency constraint on the graphs. 
     
     
         10 . The method of  claim 1 , wherein deriving comprises
 formulating an objective function encompassing smoothness and consistency constraints and providing label information in the target domain at least responsive to label information in the source domain;   applying the objective function to the all examples, whether labeled or unlabeled, and all features in order to obtain at least one result relative to the unlabeled examples;   iteratively refining the objective function to yield a label function;   providing output labels responsive to the label function.   
     
     
         11 . The method of  claim 10 , wherein iteratively refining comprises
 using normalized affinity matrices for the tripartite and bipartite graphs, evaluating the label function on the unlabeled examples by:
 propagating label information through the features to the unlabeled examples; and 
 combining with prior information for the unlabeled examples 
   using the graph structures, evaluating the label function on the features by:
 propagating label information from the labeled examples to the features; 
 propagating label information calculated in accordance with the label function from previously unlabeled examples to the features; and 
 combining with prior information for the features. 
   
     
     
         12 . The method of  claim 10 , wherein the objective function is a weighted combination of label smoothness on the tripartite graph, label smoothness on the bipartite graph, and consistency with the label information and the prior knowledge. 
     
     
         13 . The method of  claim 9 , wherein the label function yields a positive or negative result for unlabelled examples. 
     
     
         14 . A computer program product for carrying out operations, the computer program product comprising a storage medium readable by a processing circuit and storing instructions to be run by the processing circuit for performing a method comprising:
 maintaining machine readable embodiments on a medium of first and second graphs,
 the first graph comprising
 a first plurality of nodes corresponding to labeled and unlabeled examples from source and target domains; 
 a second plurality of nodes corresponding to features; and 
 a first plurality of edges connecting
 the nodes corresponding to the features 
 to the nodes corresponding to the examples according to whether the features appear in the examples or not; 
 
 
 the second graph comprising
 the first plurality of nodes corresponding to the examples; and 
 a second plurality of edges connecting the examples, the edges being associated with indications that indicate whether connected examples are in a same domain or not; 
 
   deriving labels for at least one target domain based on the first and second graphs; and   presenting an embodiment of the labels as a result.   
     
     
         15 . The program product of  claim 14 , wherein the labels in the source domain are related to a first field of application, while the labels in the target domain are related to a second field of application. 
     
     
         16 . The program product of  claim 14 , wherein deriving takes into account labels from both the target and source domains, but weights the labels from the target domain more heavily in determining derived labels for unlabelled examples in the target domain. 
     
     
         17 . The program product of  claim 14 , wherein the examples are machine readable embodiments—in a medium—of documents; and the features are machine embodiments—in a medium—of words within the documents. 
     
     
         18 . The program product of  claim 14 , wherein deriving comprises
 formulating an objective function encompassing smoothness and consistency constraints and providing label information in the target domain at least responsive to label information in the source domain;   applying the objective function to the all examples, whether labeled or unlabeled, and all features in order to obtain at least one result relative to the unlabeled examples;   iteratively refining the objective function to yield a label function;   providing output labels responsive to the label function.   
     
     
         19 . The program product of  claim 18 , wherein iteratively refining comprises
 using normalized affinity matrices for the tripartite and bipartite graphs, evaluating the label function on the unlabeled examples by:
 propagating label information through the features to the unlabeled examples; and 
 combining with prior information for the unlabeled examples 
   using the graph structures, evaluating the label function on the features by:
 propagating label information from the labeled examples to the features; 
 propagating label information calculated in accordance with the label function from previously unlabeled examples to the features; and 
 combining with prior information for the features. 
   
     
     
         20 . The program product of  claim 14 , wherein the objective function is a weighted combination of label smoothness on the tripartite graph, label smoothness on the bipartite graph, and consistency with the label information and the prior knowledge. 
     
     
         21 . A system comprising:
 at least one medium for storing data and program code;   at least one processor for performing operations in conjunction with the medium, the operations comprising
 maintaining machine readable embodiments on a medium of first and second graphs,
 the first graph comprising
 a first plurality of nodes corresponding to labeled and unlabeled examples from source and target domains; 
 a second plurality of nodes corresponding to features; and 
 a first plurality of edges connecting 
  the nodes corresponding to the features 
  to the nodes corresponding to the examples according to whether the features appear in the examples or not; 
 
 the second graph comprising
 the first plurality of nodes corresponding to the examples; and 
 a second plurality of edges connecting the examples, the edges being associated with indications that indicate whether connected examples are in a same domain or not; 
 
 
   deriving labels for at least one target domain based on the first and second graphs; and   presenting an embodiment of the labels as a result.   
     
     
         22 . The system of  claim 21 , wherein deriving takes into account labels from both the target and source domains, but weights the labels from the target domain more heavily in determining derived labels for unlabelled examples in the target domain. 
     
     
         23 . The system of  claim 21 , wherein deriving comprises
 formulating an objective function encompassing smoothness and consistency constraints and providing label information in the target domain at least responsive to label information in the source domain;   applying the objective function to the all examples, whether labeled or unlabeled, and all features in order to obtain at least one result relative to the unlabeled examples;   iteratively refining the objective function to yield a label function;   providing output labels responsive to the label function.   
     
     
         24 . The system of  claim 23 , wherein iteratively refining comprises:
 using normalized affinity matrices for the tripartite and bipartite graphs, evaluating the label function on the unlabeled examples by:
 propagating label information through the features to the unlabeled examples; and 
 combining with prior information for the unlabeled examples 
   using the graph structures, evaluating the label function on the features by:
 propagating label information from the labeled examples to the features; 
 propagating label information calculated in accordance with the label function from previously unlabeled examples to the features; and 
 combining with prior information for the features. 
   
     
     
         25 . The system of  claim 20 , wherein deriving takes into account labels from both the target and source domains, but weights the labels from the target domain more heavily in determining derived labels for unlabelled examples in the target domain.