Latent factor dependency structure determination

Abstract

Disclosed is a general learning framework for computer implementation that induces sparsity on the undirected graphical model imposed on the vector of latent factors. A latent factor model SLFA is disclosed as a matrix factorization problem with a special regularization term that encourages collaborative reconstruction. Advantageously, the model may simultaneously learn the lower-dimensional representation for data and model the pairwise relationships between latent factors explicitly. An on-line learning algorithm is disclosed to make the model amenable to large-scale learning problems. Experimental results on two synthetic data and two real-world data sets demonstrate that pairwise relationships and latent factors learned by the model provide a more structured way of exploring high-dimensional data, and the learned representations achieve the state-of-the-art classification performance.

The invention claimed is: 
     
       1. A computer implemented method of structured latent factor analysis comprising:
 by a computer: 
 learning one or more hidden dependency structures of latent factors of a set of data; 
 modeling pairwise relationships among them and determining structural relationships through the use of a sparse Gaussian graphical model; 
 outputting an indication of the latent factor relationships; 
 wherein said pairwise relationship modeling is performed according to the following pairwise Markov Random Field (MRF) prior on a vector of factors sε   K : 
 
       
         
           
             
               
                 
                   
                     p 
                     ( 
                     
                       
                         s 
                         ⁢ 
                         
                            
                           
                             μ 
                             , 
                             Θ 
                           
                           ) 
                         
                       
                       = 
                       
                         
                           1 
                           
                             Z 
                             ⁡ 
                             
                               ( 
                               
                                 μ 
                                 , 
                                 Θ 
                               
                               ) 
                             
                           
                         
                         ⁢ 
                         
                           exp 
                           ⁡ 
                           
                             ( 
                             
                               
                                 - 
                                 
                                   
                                     ∑ 
                                     
                                       i 
                                       = 
                                       1 
                                     
                                     K 
                                   
                                   ⁢ 
                                   
                                     
                                       μ 
                                       i 
                                     
                                     ⁢ 
                                     
                                       s 
                                       i 
                                     
                                   
                                 
                               
                               - 
                               
                                 
                                   1 
                                   2 
                                 
                                 ⁢ 
                                 
                                   
                                     ∑ 
                                     
                                       i 
                                       = 
                                       1 
                                     
                                     K 
                                   
                                   ⁢ 
                                   
                                     
                                       ∑ 
                                       
                                         j 
                                         = 
                                         1 
                                       
                                       K 
                                     
                                     ⁢ 
                                     
                                       
                                         θ 
                                         ij 
                                       
                                       ⁢ 
                                       
                                         s 
                                         i 
                                       
                                       ⁢ 
                                       
                                         s 
                                         j 
                                       
                                     
                                   
                                 
                               
                             
                             ) 
                           
                         
                       
                     
                   
                 
                 
                   
                     ( 
                     4 
                     ) 
                   
                 
               
             
           
         
          with parameter μ=[μi], symmetric Θ=[θ ij ], and partition function Z(μ,Θ) which normalizes the distribution, wherein p is a probability of a field configuration of (s|μ, Θ), K is a number of latent factors, s is an element of natural parameter    K , and i and j are non-zero variables; and 
         modeling the pairwise interaction simultaneously with the learning one or more hidden dependency structures of latent factors of a set of data. 
       
     
     
       2. The computer implemented method of  claim 1  wherein said model identifies a dependency structure in the latent space. 
     
     
       3. The computer implemented method of  claim 1  wherein said model is determined by a unidirected graphical model.