Multivariate data mixture model estimation device, mixture model estimation method, and mixture model estimation program

Abstract

With respect to the model selection issue of a mixture model, the present invention performs high-speed model selection under an appropriate standard regarding the number of model candidates which exponentially increases as the number and the types to be mixed increase. A mixture model estimation device comprises: a data input unit to which data of a mixture model to be estimated, candidate values of the number of mixtures which are required for estimating the mixture model of the data, and types of components configuring the mixture model and parameters thereof, are input; a processing unit which sets the number of mixtures from the candidate values, calculates, with respect to the set number of mixtures, a variation probability of a hidden variable for a random variable which becomes a target for mixture model estimation of the data, and estimates the optimal mixture model by optimizing the types of the components and the parameters therefor using the calculated variation probability of the hidden variable so that the lower bound of the posterior probabilities of the model separated for each component of the mixture model can be maximized; and a model estimation result output unit which outputs the model estimation result obtained by the processing unit.

1 . A mixture model estimation device comprising:
 a data input unit that inputs data of a mixture model to be estimated, and candidate values for a mixture number, and types and parameters of components constituting the mixture model that are necessary for estimating the mixture model of the data;   a processing unit comprising a computer hardware processor that sets the mixture number from the candidate values, calculates a variation probability of a hidden variable for a random variable which is a target for estimating the mixture model of the data with respect to the set mixture number, and optimally estimates the mixture model by optimizing the types and parameters of the components using the calculated variation probability of the hidden variable so that a lower bound of a model posterior probability separated for each of the components of the mixture model is maximized; and   a model estimation result output unit that outputs a model estimation result obtained by the processing unit.   
     
     
         2 . The mixture model estimation device according to  claim 1 , wherein the processing unit obtains the mixture number of the mixture model optimally by calculating the lower bound of the model posterior probability and the types and parameters of the components for all the candidate values for the mixture number. 
     
     
         3 . The mixture model estimation device according to  claim 1 , wherein:
 the mixture number is denoted by C, the random variable is denoted by X, the types of the components are denoted by S 1 , . . . , S C , and the parameters of the components are denoted by θ=(π 1 , . . . , π C , φ 1   S1 , . . . , φ C   SC ) (π 1 , . . . , π C  are mixture ratios when the mixture number is 1 to C, and φ 1   S1 , . . . , φ C   SC  are parameters of distributions of components S 1  to S C  when the mixture number is 1 to C),   the mixture model is expressed by equation 1:   
       
         
           
             
               
                 
                   
                     [ 
                     
                       Math 
                       . 
                       
                           
                       
                        
                       1 
                     
                     ] 
                   
                 
                 
                   
                       
                   
                 
               
               
                 
                   
                     
                       P 
                        
                       
                         ( 
                         
                           X 
                           | 
                           θ 
                         
                         ) 
                       
                     
                     = 
                     
                       
                         ∑ 
                         
                           c 
                           = 
                           1 
                         
                         C 
                       
                        
                       
                         
                           π 
                           c 
                         
                          
                         
                           
                             P 
                             c 
                           
                            
                           
                             ( 
                             
                               X 
                               ; 
                               
                                 φ 
                                 c 
                                 
                                   S 
                                   c 
                                 
                               
                             
                             ) 
                           
                         
                       
                     
                   
                 
                 
                   
                     ( 
                     1 
                     ) 
                   
                 
               
             
           
         
         when the hidden variable for the random variable X is denoted by Z=(Z 1 , . . . , Z C ), a joint distribution of a complete variable that is a pair of the random variable X and the hidden variable Z is defined by equation 2: 
       
       
         
           
             
               
                 
                   
                     [ 
                     
                       Math 
                       . 
                       
                           
                       
                        
                       2 
                     
                     ] 
                   
                 
                 
                   
                       
                   
                 
               
               
                 
                   
                     
                       P 
                        
                       
                         ( 
                         
                           X 
                           , 
                           
                             Z 
                             | 
                             θ 
                           
                         
                         ) 
                       
                     
                     = 
                     
                       
                         ∑ 
                         
                           c 
                           = 
                           1 
                         
                         C 
                       
                        
                       
                         
                           ( 
                           
                             
                               π 
                               c 
                             
                              
                             
                               
                                 P 
                                 c 
                               
                                
                               
                                 ( 
                                 
                                   X 
                                   ; 
                                   
                                     φ 
                                     c 
                                     
                                       S 
                                       c 
                                     
                                   
                                 
                                 ) 
                               
                             
                           
                           ) 
                         
                         
                           Z 
                           c 
                         
                       
                     
                   
                 
                 
                   
                     ( 
                     2 
                     ) 
                   
                 
               
             
           
         
         when N data values of the random variable X are denoted by x n  (n=1, . . . , N), and N values of the hidden variable Z for the values x n  are denoted by z n  (n=1, . . . , N), a posterior probability of the hidden variable Z is expressed by equation 3:
   [Math. 3] 
     P ( z   n   |x   n ,θ)∝π c   P   c ( x   n ;φ c   S     c   )  (3)
 
 
       
     
     
         4 . The mixture model estimation device according to  claim 1 , wherein the mixture model comprises a plurality of mixture distributions having different independent characteristics. 
     
     
         5 . The mixture model estimation device according  claim 1 , wherein the mixture model comprises a plurality of various mixture distributions. 
     
     
         6 . The mixture model estimation device according to  claim 1 , wherein the mixture model comprises mixture distributions of different stochastic regression functions. 
     
     
         7 . The mixture model estimation device according to  claim 1 , wherein the mixture model comprises mixture distributions of different stochastic discriminant functions. 
     
     
         8 . The mixture model estimation device according to  claim 1 , wherein the mixture model comprises mixture distributions of a hidden Markov model having different output probabilities. 
     
     
         9 . A mixture model estimation method comprising:
 by using an input unit, inputting data of a mixture model to be estimated, and candidate values for a mixture number, and types and parameters of components constituting the mixture model that are necessary for estimating the mixture model of the data;   causing a processing unit to set the mixture number from the candidate values, calculate a variation probability of a hidden variable for a random variable which is a target for estimating the mixture model of the data, and optimally estimate the mixture model by optimizing the types and parameters of the components using the calculated variation probability of the hidden variable so that a lower bound of a model posterior probability separated for each of the components of the mixture model is maximized; and   causing a model estimation result output unit to output a model estimation result obtained by the processing unit.   
     
     
         10 . In the mixture model estimation method of  claim 9 , the processing unit obtains the mixture number of the mixture model optimally by calculating the lower bound of the model posterior probability and the types and parameters of the components for all the candidate values for the mixture number. 
     
     
         11 . In the mixture model estimation method of  claim 9 , wherein:
 the mixture number is denoted by C, the random variable is denoted by X, the types of the components are denoted by S 1 , . . . , S C , and the parameters of the components are denoted by θ=(π 1 , . . . , π C , φ 1   S1 , . . . , φ C   SC ) (π 1 , . . . , π C  are mixture ratios when the mixture number is 1 to C, and φ 1   S1 , . . . , φ C   SC  are parameters of distributions of components S 1  to S C  when the mixture number is 1 to C), the mixture model is expressed by equation 1:   
       
         
           
             
               
                 
                   
                     [ 
                     
                       Math 
                       . 
                       
                           
                       
                        
                       1 
                     
                     ] 
                   
                 
                 
                   
                       
                   
                 
               
               
                 
                   
                     
                       P 
                        
                       
                         ( 
                         
                           X 
                           | 
                           θ 
                         
                         ) 
                       
                     
                     = 
                     
                       
                         ∑ 
                         
                           c 
                           = 
                           1 
                         
                         C 
                       
                        
                       
                         
                           π 
                           c 
                         
                          
                         
                           
                             P 
                             c 
                           
                            
                           
                             ( 
                             
                               X 
                               ; 
                               
                                 φ 
                                 c 
                                 
                                   S 
                                   c 
                                 
                               
                             
                             ) 
                           
                         
                       
                     
                   
                 
                 
                   
                     ( 
                     1 
                     ) 
                   
                 
               
             
           
         
         the hidden variable for the random variable X is denoted by Z=(Z 1 , . . . , Z C ), a joint distribution of a complete variable that is a pair of the random variable X and the hidden variable X is defined by equation 2: 
       
       
         
           
             
               
                 
                   
                     [ 
                     
                       Math 
                       . 
                       
                           
                       
                        
                       2 
                     
                     ] 
                   
                 
                 
                   
                       
                   
                 
               
               
                 
                   
                     
                       P 
                        
                       
                         ( 
                         
                           X 
                           , 
                           
                             Z 
                             | 
                             θ 
                           
                         
                         ) 
                       
                     
                     = 
                     
                       
                         ∑ 
                         
                           c 
                           = 
                           1 
                         
                         C 
                       
                        
                       
                         
                           ( 
                           
                             
                               π 
                               c 
                             
                              
                             
                               
                                 P 
                                 c 
                               
                                
                               
                                 ( 
                                 
                                   X 
                                   ; 
                                   
                                     φ 
                                     c 
                                     
                                       S 
                                       c 
                                     
                                   
                                 
                                 ) 
                               
                             
                           
                           ) 
                         
                         
                           Z 
                           c 
                         
                       
                     
                   
                 
                 
                   
                     ( 
                     2 
                     ) 
                   
                 
               
             
           
         
         if N data values of the random variable X are denoted by X n  (n=1, . . . , N), and N values of the hidden variable Z for the values X n  are denoted by Z n  (n=1, . . . , N), a posterior probability of the hidden variable Z is expressed by equation 3:
   [Math. 3] 
     P ( z   n   |x   n ,θ)∝π c   P   c ( x   n ;φ c   S     c   )  (3)
 
 
       
     
     
         12 . In the mixture model estimation method of  claim 9 , the mixture model includes a plurality of mixture distributions having different independence characteristics. 
     
     
         13 . In the mixture model estimation method of  claim 9 , the mixture model includes a plurality of various mixture distributions. 
     
     
         14 . In the mixture model estimation method of  claim 9 , the mixture model includes mixture distributions of different stochastic regression functions. 
     
     
         15 . In the mixture model estimation method of  claim 9 , the mixture model includes mixture distributions of different stochastic discriminant functions. 
     
     
         16 . In the mixture model estimation method of  claim 9 , the mixture model includes mixture distributions of a hidden Markov model having different output probabilities. 
     
     
         17 . A non-transitory computer-readable medium storing a computer-readable mixture model estimation program for operating a computer as a mixture model estimation device comprising:
 an input unit that inputs data of a mixture model to be estimated, and candidate values for a mixture number, and types and parameters of components constituting the mixture model that are necessary for estimating the mixture model of the data;   a processing unit comprising a processor that sets the mixture number from the candidate values, calculates a variation probability of a hidden variable for a random variable which is a target for estimating the mixture model of the data with respect to the set mixture number, and optimally estimates the mixture model by optimizing the types and parameters of the components using the calculated variation probability of the hidden variable so that a lower bound of a model posterior probability separated for each of the components of the mixture model is maximized; and   a model estimation result output unit that outputs a model estimation result obtained by the processing unit.   
     
     
         18 . In the mixture model estimation program of  claim 17 , the optimal mixture number of the mixture model is optimally obtained by calculating the lower bound of the model posterior probability and the types and parameters of the components for all the candidate values for the mixture number. 
     
     
         19 . In the mixture model estimation program of  claim 17 , wherein
 the mixture number is denoted by C, the random variable is denoted by X, the types of the components are denoted by S 1 , . . . , S C , and the parameters of the components are denoted by θ=(π 1 , . . . , π C , φ 1   S1 , . . . , φ C   SC ) (π 1 , . . . , π C  are mixture ratios when the mixture number is 1 to C, and φ 1   S1 , . . . , φ C   SC  are parameters of distributions of components S 1  to S C  when the mixture number is 1 to C), the mixture model is expressed by equation 1:   
       
         
           
             
               
                 
                   
                     [ 
                     
                       Math 
                       . 
                       
                           
                       
                        
                       1 
                     
                     ] 
                   
                 
                 
                   
                       
                   
                 
               
               
                 
                   
                     
                       P 
                        
                       
                         ( 
                         
                           X 
                           | 
                           θ 
                         
                         ) 
                       
                     
                     = 
                     
                       
                         ∑ 
                         
                           c 
                           = 
                           1 
                         
                         C 
                       
                        
                       
                         
                           π 
                           c 
                         
                          
                         
                           
                             P 
                             c 
                           
                            
                           
                             ( 
                             
                               X 
                               ; 
                               
                                 φ 
                                 c 
                                 
                                   S 
                                   c 
                                 
                               
                             
                             ) 
                           
                         
                       
                     
                   
                 
                 
                   
                     ( 
                     1 
                     ) 
                   
                 
               
             
           
         
         when the hidden variable for the random variable X is denoted by Z=(Z 1 , . . . , Z C ), a joint distribution of a complete variable that is a pair of the random variable X and the hidden variable X is defined by equation 2: 
       
       
         
           
             
               
                 
                   
                     [ 
                     
                       Math 
                       . 
                       
                           
                       
                        
                       2 
                     
                     ] 
                   
                 
                 
                   
                       
                   
                 
               
               
                 
                   
                     
                       P 
                        
                       
                         ( 
                         
                           X 
                           , 
                           
                             Z 
                             | 
                             θ 
                           
                         
                         ) 
                       
                     
                     = 
                     
                       
                         ∑ 
                         
                           c 
                           = 
                           1 
                         
                         C 
                       
                        
                       
                         
                           ( 
                           
                             
                               π 
                               c 
                             
                              
                             
                               
                                 P 
                                 c 
                               
                                
                               
                                 ( 
                                 
                                   X 
                                   ; 
                                   
                                     φ 
                                     c 
                                     
                                       S 
                                       c 
                                     
                                   
                                 
                                 ) 
                               
                             
                           
                           ) 
                         
                         
                           Z 
                           c 
                         
                       
                     
                   
                 
                 
                   
                     ( 
                     2 
                     ) 
                   
                 
               
             
           
         
         when N data values of the random variable X are denoted by X n  (n=1, . . . , N), and N values of the hidden variable Z for the values X n  are denoted by Z n  (n=1, . . . , N), a posterior probability of the hidden variable Z is expressed by equation 3:
   [Math. 3] 
     P ( z   n   |x   n ,θ)∝π c   P   c ( x   n ;φ c   S     c   )  (3)
 
 
       
     
     
         20 . The mixture model estimation device according to  claim 1 , wherein the processing unit calculates the variation probability of the hidden variable by solving an optimization problem expressed by a first equation, calculates the lower bound of the model posterior probability by a second equation, calculates an optimal mixture model H (t)  and parameters θ (t)  of components of the optimal mixture model after t iterations by using the variation probability of the hidden variable by a third equation, and determines whether the lower bound of the model posterior probability converges by using a fourth equation, wherein when the processing unit determines that the lower bound of the model posterior probability does not converge, the processing unit repeats processes of first to fourth equation, and if the processing unit determines that the lower bound converges, the processing unit compares a lower bound of a model posterior probability of a currently-set optimal mixture model with the lower bound of the model posterior probability obtained through calculations, and sets the larger value as the optimal mixture model, and the processing unit repeats the processes of first to fourth equation for all the candidate values for the mixture number so as to estimate the mixture model optimally.