Semi-supervised learning based on semiparametric regularization

Abstract

Semi-supervised learning plays an important role in machine learning and data mining. The semi-supervised learning problem is approached by developing semiparametric regularization, which attempts to discover the marginal distribution of the data to learn the parametric function through exploiting the geometric distribution of the data. This learned parametric function can then be incorporated into the supervised learning on the available labeled data as the prior knowledge. A semi-supervised learning approach is provided which incorporates the unlabeled data into the supervised learning by a parametric function learned from the whole data including the labeled and unlabeled data. The parametric function reflects the geometric structure of the marginal distribution of the data. Furthermore, the proposed approach which naturally extends to the out-of-sample data is an inductive learning method in nature.

What is claimed is: 
     
       1. A semisupervised learning method, comprising:
 analyzing a data set using at least one automated processor, comprising labeled data and unlabeled data, by performing a principal component analysis to derive parameters of a parametric function of the feature space reflecting a geometric structure of a marginal distribution of the data set according to its principal components; 
 performing supervised learning on the labeled data using the at least one automated processor, based on the parametric function of the feature space reflecting the geometric structure of the marginal distribution of the entire data set; and 
 storing information derived from said supervised learning in a computer memory, 
 wherein the parametric function is dependent on both the data set and said principal component analysis. 
 
     
     
       2. The method according to  claim 1 , wherein the analyzing is performed through a parametric function by principal component analysis in a Reproducing Kernel Hilbert Space, and the supervised learning is performed based on the labeled data in the Reproducing Kernel Hilbert Space, the Reproducing Kernel Hilbert Space being extended by including the parametric function derived based on the entire data set. 
     
     
       3. The method according to  claim 1 , further comprising classifying unlabeled data based on the stored supervised learning information. 
     
     
       4. The method according to  claim 1 , further comprising performing a binary classification of unlabeled data based on the stored supervised learning information. 
     
     
       5. The method according to  claim 1 , wherein data points projected onto a principal component axis maintain their geometric relationship in the feature space with other data points. 
     
     
       6. The method according to  claim 1 , wherein the principal component analysis comprises a kernel principal component analysis. 
     
     
       7. An apparatus for performing semisupervised learning on a data set, comprising:
 a memory adapted to store a data set, comprising labeled data and unlabeled data; 
 at least one automated processor, configured to analyze the data set through a parametric function derived by principal component analysis of the feature space reflecting a geometric structure of a marginal distribution of the data set according to its principal components, and performing supervised learning on the labeled data based on the parametric function derived by principal component analysis of the feature space reflecting the geometric structure of the entire data set; and 
 a memory adapted to store information derived from said supervised learning in a computer memory. 
 
     
     
       8. The apparatus according to  claim 7 , wherein the principal component analysis is performed in a Reproducing Kernel Hilbert Space, and the supervised learning is performed based on the labeled data in the Reproducing Kernel Hilbert Space, the Reproducing Kernel Hilbert Space being extended by including the parametric function derived based on the entire data set. 
     
     
       9. The apparatus according to  claim 7 , wherein the automated processor classifies unlabeled data based on the stored supervised learning information. 
     
     
       10. The apparatus according to  claim 7 , wherein the automated processor performs a binary classification of unlabeled data based on the stored supervised learning information. 
     
     
       11. The apparatus according to  claim 7 , wherein data points projected onto a principal component axis maintain their geometric relationship in the feature space with other data points. 
     
     
       12. The apparatus according to  claim 7 , wherein the principal component analysis comprises a kernel principal component analysis. 
     
     
       13. A method, comprising:
 storing a data set comprising both labeled and unlabeled data; 
 analyzing, with at least one automated processor, the entire data set using a statistical analysis of variance within the feature space of the data set to determine a geometric structure of the data set dependent on the statistical analysis of variance, by performing at least one orthogonal linear transform; 
 analyzing, with the at least one automated processor, the labeled data in dependence on the determined geometric structure of the data set dependent on the statistical analysis of variance, to learn at least one classification criterion from the classification and features of the labeled data; and 
 automatically classifying unlabeled data based on the learned classification criterion. 
 
     
     
       14. The method according to  claim 13 , further comprising classifying at least one unlabeled data point. 
     
     
       15. The method according to  claim 13 , further comprising classifying at least one data point outside of the data set. 
     
     
       16. The method according to  claim 13 , wherein the analyzing performs a kernel principal component analysis of the data set. 
     
     
       17. The method according to  claim 16 , wherein the kernel principal component analysis is performed in a Reproducing Kernel Hilbert Space. 
     
     
       18. The method according to  claim 17 , wherein the Reproducing Kernel Hilbert Space is extended by including the determined geometric structure based on the entire data set. 
     
     
       19. The method according to  claim 13 , wherein a projection of the data is determined which maximizes a variance of features within the feature space. 
     
     
       20. The method according to  claim 13 , further comprising analyzing eigenvalues of the entire data set, including both labeled and unlabeled data.