Open vs enclosed spatial environment classification for a mobile or wearable device using microphone and deep learning method

Abstract

A method and apparatus for classifying a spatial environment as open or enclosed are provided. In the method and apparatus, one or more microphones detect ambient sound in a spatial environment and output an audio signal representative of the ambient sound. A processor determines a spatial environment impulse response (SEIR) for the audio signal and extracts one or more features of the SEIR. The processor classifies the spatial environment as open or enclosed based on the one or more features of the SEIR.

The invention claimed is: 
     
       1. A method, comprising:
 detecting, by one or more microphones, ambient sound in a spatial environment; 
 outputting, to a processor, an audio signal representative of the ambient sound; 
 determining, by the processor, a spatial environment impulse response (SEIR) for the audio signal; 
 extracting one or more features of the SEIR; and 
 classifying, by a pattern classifier executed by the processor, the spatial environment as open or enclosed based on the one or more features of the SEIR. 
 
     
     
       2. The method of  claim 1 , wherein:
 determining the SEIR for the audio signal includes:
 performing a deconvolution on the audio signal; and 
 determining a cepstrum for the deconvoluted audio signal; 
 
 the method comprises:
 augmenting the one or more features of the SEIR with features extracted from Mel-Frequency Cepstral Coefficients (MFCCs), delta MFCCs or double delta MFCC to form a composite vector; and 
 
 classifying the spatial environment as open or enclosed includes:
 classifying the spatial environment as open or enclosed based on the composite vector. 
 
 
     
     
       3. The method of  claim 1 , wherein classifying the spatial environment as open or enclosed includes identifying a type of the spatial environment as office, home, mall, supermarket, street, stadium, beach or nature. 
     
     
       4. The method of  claim 1 , wherein determining the SEIR for the audio signal includes:
 dividing the audio signal into a plurality of frames; 
 determining an energy ratio for each frame of the plurality of frames; 
 selecting, from the plurality of frames, a set of frames having respective energy ratios that meet a criterion; 
 performing exponential windowing on the set of frames to minimize phase; 
 determining a cepstrum for the set of frames; and 
 performing inverse exponential windowing on the set of frames. 
 
     
     
       5. The method of  claim 1 , wherein extracting the one or more features of the SEIR includes:
 obtaining a first SEIR feature of the one or more features as energy of multiple bands of initial samples of the SEIR; and 
 obtaining a second SEIR feature of the one or more features as an average of maxima indices of SEIR magnitude. 
 
     
     
       6. The method of  claim 5 , wherein extracting the one or more features of the SEIR includes:
 obtaining a third SEIR feature of the one or more features as a time kurtosis of the SEIR; 
 obtaining a fourth SEIR feature of the one or more features as a spectral standard deviation at a center frequency of the SEIR; and 
 obtaining a fifth SEIR feature of the one or more features as a slope of samples of the SEIR. 
 
     
     
       7. The method of  claim 2 , comprising:
 performing cepstral mean subtraction on the features extracted from the MFCC, delta MFCC or double delta MFCC to reduce mismatch between training and testing conditions. 
 
     
     
       8. A device, comprising:
 one or more microphones configured to:
 detect ambient sound in a spatial environment; and 
 output an audio signal representative of the ambient sound; and 
 
 a processor configured to:
 receive the audio signal representative of the ambient sound; 
 determine a spatial environment impulse response (SEIR) for the audio signal; 
 extract one or more features of the SEIR; and 
 classify the spatial environment as open or enclosed based on the one or more features of the SEIR. 
 
 
     
     
       9. The device of  claim 8 , wherein the processor is configured to:
 determine the SEIR for the audio signal by at least:
 performing a deconvolution on the audio signal; and 
 determining a cepstrum for the deconvoluted audio signal; 
 
 augment the one or more features of the SEIR with features extracted from Mel-frequency cepstral coefficients (MFCCs), delta MFCCs or double delta MFCC to form a composite vector; and 
 classify the spatial environment as open or enclosed by classifying the spatial environment as open or enclosed based on the composite vector. 
 
     
     
       10. The device of  claim 8 , wherein classifying the spatial environment as open or enclosed includes identifying a type of the spatial environment as office, home, mall, supermarket, street, stadium, beach or nature. 
     
     
       11. The device of  claim 8 , wherein the processor is configured to determine the SEIR for the audio signal by:
 dividing the audio signal into a plurality of frames; 
 determining an energy ratio for each frame of the plurality of frames; 
 selecting, from the plurality of frames, a set of frames having respective energy ratios that meet a criterion; 
 performing exponential windowing on the set of frames to minimize phase; 
 determining a cepstrum for the set of frames; and 
 performing inverse exponential windowing on the set of frames. 
 
     
     
       12. The device of  claim 8 , wherein the processor is configured to extract the one or more features of the SEIR by:
 obtaining a first SEIR feature of the one or more features as energy of multiple bands of initial samples of the SEIR; and 
 obtaining a second SEIR feature of the one or more features as an average of maxima indices of SEIR magnitude. 
 
     
     
       13. The device of  claim 12 , wherein the processor is configured to extract the one or more features of the SEIR by:
 obtaining a third SEIR feature of the one or more features as a time kurtosis of the SEIR; 
 obtaining a fourth SEIR feature of the one or more features as a spectral standard deviation at a center frequency of the SEIR; and 
 obtaining a fifth SEIR feature of the one or more features as a slope of samples of the SEIR. 
 
     
     
       14. The device of  claim 9 , wherein the processor is configured to:
 perform cepstral mean subtraction on the features extracted from the MFCC, delta MFCC or double delta MFCC to reduce mismatch between training and testing conditions. 
 
     
     
       15. A system, comprising:
 a processor; and 
 memory configured to store executable instructions that, when executed by the processor, cause the processor to:
 receive an audio signal representative of ambient sound of a spatial environment; 
 determine a spatial environment impulse response (SEIR) for the audio signal; 
 extract one or more features of the SEIR; and 
 classify the spatial environment as open or enclosed based on the one or more features of the SEIR. 
 
 
     
     
       16. The system of  claim 15 , wherein the executable instructions cause the processor to:
 determine the SEIR for the audio signal by at least:
 performing a deconvolution on the audio signal; and 
 determining a cepstrum for the deconvoluted audio signal; 
 
 augment the one or more features of the SEIR with features extracted from Mel-frequency cepstral coefficients (MFCCs), delta MFCCs or double delta MFCC to form a composite vector; and 
 classify the spatial environment as open or enclosed by at least:
 classifying the spatial environment as open or enclosed based on the composite vector. 
 
 
     
     
       17. The system of  claim 15 , wherein classifying the spatial environment as open or enclosed includes identifying a type of the spatial environment as office, home, mall, supermarket, street, stadium, beach or nature. 
     
     
       18. The system of  claim 15 , wherein the executable instructions cause the processor to classify the spatial environment as open or enclosed by at least:
 dividing the audio signal into a plurality of frames; 
 determining an energy ratio for each frame of the plurality of frames; 
 selecting, from the plurality of frames, a set of frames having respective energy ratios that meet a criterion; 
 performing exponential windowing on the set of frames to minimize phase; 
 determining a cepstrum for the set of frames; and 
 performing inverse exponential windowing on the set of frames. 
 
     
     
       19. The system of  claim 15 , wherein the executable instructions cause the processor to extract the one or more features of the SEIR by:
 obtaining a first SEIR feature of the one or more features as energy of multiple bands of initial samples of the SEIR; and 
 obtaining a second SEIR feature of the one or more features as an average of maxima indices of SEIR magnitude. 
 
     
     
       20. The system of  claim 19 , wherein the executable instructions cause the processor to extract the one or more features of the SEIR by:
 obtaining a third SEIR feature of the one or more features as a time kurtosis of the SEIR; 
 obtaining a fourth SEIR feature of the one or more features as a spectral standard deviation at a center frequency of the SEIR; and 
 obtaining a fifth SEIR feature of the one or more features as a slope of samples of the SEIR.