Data training in multi-sensor setups

Abstract

A system and method for constructing training dictionaries with multichannel information. An exemplary method takes into account the effect of the acoustic path while training multichannel acoustic data. A method that uses different time-frequency resolutions in machine learning training is also presented.

1 - 20 . (canceled) 
     
     
         21 . A method to improve the separation of audio sources comprising:
 obtaining from a sensor a first time series of N audio signal samples, s1, s2, . . . , sN, corresponding to a known training signal;   transforming the time series of N samples to a first spectrogram;   wherein the first spectrogram comprises a first number of frequency bins and a first number of time frames;   transforming the same time series of N samples, s1, s2, . . . , sN, to a second spectrogram;   wherein the second spectrogram comprises a second number of frequency bins and a second number of time frames, different from the first number of time frames;   mapping the first spectrogram to a third spectrogram through a matrix multiplication such that the third spectrogram comprises a third number of frequency bins and a third number of time frames, equal to the first number of time frames;   mapping the second spectrogram to a fourth spectrogram through a matrix multiplication such that the fourth spectrogram comprises the same third number of frequency bins and a fourth number of time frames, equal to the second number of time frames;   wherein the third number of frequency bins is different than the first number of frequency bins and the third number of frequency bins is different than the second number of frequency bins;   determining elements of a training dictionary associated with the third and fourth spectrograms;   storing the training dictionary elements;   using the training dictionary elements to process a second time series of audio signal samples obtained by the sensor; and   audibly outputting an audio signal related to the processed second series of audio signal samples.   
     
     
         22 . The method of  claim 21 , wherein the first and second number of frequency bins are different. 
     
     
         23 . The method of  claim 21 , where the source signal is single channel or binaural or multichannel audio signal. 
     
     
         24 . The method of  claim 21 , wherein the determining of the training elements comprises one or more signal processing algorithms, where the one or more signal processing algorithms are one or more of non-negative matrix factorization, non-negative tensor factorization, independent component analysis, principal component analysis, singular value decomposition, dependent component analysis, low-complexity coding and decoding, stationary subspace analysis, common spatial pattern, empirical mode decomposition, tensor decomposition, canonical polyadic decomposition, higher-order singular value decomposition, and tucker decomposition. 
     
     
         25 . The method of  claim 21 , where the training dictionary is used for source separation. 
     
     
         26 . The method of  claim 21 , where the representations of the time series of N samples can be obtained with any one or more of a short-time Fourier transform (STFT), a wavelet transform, a polyphase filterbank, a multi rate filterbank, a quadrature mirror filterbank, a warped filterbank, an auditory-inspired filterbank, and a tree-structured array of filterbanks. 
     
     
         27 . The method of  claim 21 , wherein the data are captured in live or studio music events from one or more microphones. 
     
     
         28 . The method of  claim 21 , wherein the frequency bins in the third and fourth spectrogram are non-uniform. 
     
     
         29 . A system to improve the separation of audio signals comprising:
 a sensor configured to obtain a first time series of N audio signal samples, s1, s2, . . . , sN, corresponding to a known training signal;   a first transformer configured to transform the time series of N samples to a first spectrogram;   wherein the first spectrogram comprises a first number of frequency bins and a first number of time frames;   a second transformer configured to tranthe same time series of N samples, s1, s2, . . . , sN, to a second spectrogram;   wherein the second spectrogram comprises a second number of frequency bins and a second number of time frames, different from the first number of time frames;   a processor configured to map the first spectrogram to a third spectrogram through a matrix multiplication such that the third spectrogram comprises a third number of frequency bins and a third number of time frames, equal to the first number of time frames;   said processor further configured to map the second spectrogram to a fourth spectrogram through a matrix multiplication such that the fourth spectrogram comprises the same third number of frequency bins and a fourth number of time frames, equal to the second number of time frames;   wherein the third number of frequency bins is different than the first number of frequency bins and the third number of frequency bins is different than the second number of frequency bins;   said processor further configured to determine elements of a training dictionary associated with the third and fourth spectrograms;   a storage configured to store the training dictionary elements;   said processor further configured to use the training dictionary elements to process a second time series of audio signal samples obtained by the sensor; and   a device configured to audibly output an audio signal related to the processed second series of audio signal samples.   
     
     
         30 . The system of  claim 29 , wherein the first and second number of frequency bins are different. 
     
     
         31 . The system of  claim 29 , wherein the third number of frequency bins are non-uniform. 
     
     
         32 . The system of  claim 29 , wherein the source signal is a single channel or binaural or multichannel audio signal. 
     
     
         33 . The system of  claim 29 , wherein the determining of the training elements comprises one or more signal processing algorithms, wherein the one or more signal processing algorithms are one or more of non-negative matrix factorization, non-negative tensor factorization, independent component analysis, principal component analysis, singular value decomposition, dependent component analysis, low-complexity coding and decoding, stationary subspace analysis, common spatial pattern, empirical mode decomposition, tensor decomposition, canonical polyadic decomposition, higher-order singular value decomposition, and tucker decomposition. 
     
     
         34 . The system of  claim 29 , wherein the training dictionary is used for source separation. 
     
     
         35 . The system of  claim 29 , wherein a time-frequency representation of the time series of N samples can be obtained with any one or more of a short-time Fourier transform (STFT), a wavelet transform, a polyphase filterbank, a multi rate filterbank, a quadrature mirror filterbank, a warped filterbank, an auditory-inspired filterbank, and a tree-structured array of filterbanks. 
     
     
         36 . The system of  claim 29 , wherein the frequency bins in the third and fourth spectrogram are non-uniform 
     
     
         37 . A non-transitory information storage media having stored thereon information, that when executed by one or more processors, cause to be performed a method for improving the separation of audio signals comprising:
 obtaining from a sensor a first time series of N audio signal samples, s1, s2, . . . , sN, corresponding to a known training signal;   transforming the time series of N samples to a first spectrogram;   wherein the first spectrogram comprises a first number of frequency bins and a first number of time frames;   transforming the same time series of N samples, s1, s2, . . . , sN, to a second;   wherein the second spectrogram comprises a second number of frequency bins and a second number of time frames, different from the first number of time frames;   mapping the first spectrogram to a third spectrogram through a matrix multiplication such that the third spectrogram comprises a third number of frequency bins and a third number of time frames, equal to the first number of time frames;   mapping the second spectrogram to a fourth spectrogram through a matrix multiplication such that the fourth spectrogram comprises the same third number of frequency bins and a fourth number of time frames, equal to the second number of time frames;   wherein the third number of frequency bins is different than the first number of frequency bins and the third number of frequency bins is different than the second number of frequency bins;   determining elements of a training dictionary associated with the third and fourth spectrograms;   storing the training dictionary elements;   using the training dictionary elements to process a second time series of audio signal samples obtained by the sensor; and   audibly outputting an audio signal related to the processed second series of audio signal samples.   
     
     
         38 . The media of  claim 37 , wherein the first and second number of frequency bins are different. 
     
     
         39 . The media of  claim 37 , wherein the third number of frequency bins are non-uniform. 
     
     
         40 . The media of  claim 37 , wherein the source signal is a single channel or binaural or multichannel audio signal. 
     
     
         41 . The media of  claim 37 , wherein the determining of the training elements comprises one or more signal processing algorithms, wherein the one or more signal processing algorithms are one or more of non-negative matrix factorization, non-negative tensor factorization, independent component analysis, principal component analysis, singular value decomposition, dependent component analysis, low-complexity coding and decoding, stationary subspace analysis, common spatial pattern, empirical mode decomposition, tensor decomposition, canonical polyadic decomposition, higher-order singular value decomposition, and tucker decomposition. 
     
     
         42 . The media of  claim 37 , wherein the training dictionary is used for source separation. 
     
     
         43 . The media of  claim 37 , wherein a time-frequency representation of the time series of N samples can be obtained with any one or more of a short-time Fourier transform (STFT), a wavelet transform, a polyphase filterbank, a multi rate filterbank, a quadrature mirror filterbank, a warped filterbank, an auditory-inspired filterbank, and a tree-structured array of filterbanks. 
     
     
         44 . The media of  claim 37 , wherein the frequency bins in the third and fourth spectrogram are non-uniform.