Method and system for speech detection and speech enhancement

Abstract

A method of speech detection and speech enhancement in a speech detection and speech enhancement unit of Multipoint Conferencing Node (MCN) and a method of training the same. The method comprising receiving input audio segments, and determining an acoustic environment based on input audio auxiliary information, extracting T-F-domain features from the received input audio segments, determining if each of the received input audio segments is speech by inputting the T-F domain features into a speech detection classifier trained for the determined acoustic environment, determining, when one of the received input audio segments is speech, if the received audio segment is noisy speech by inputting the T-F domain features into a noise classifier using a statistical generative model representing the probability distributions of the T-F domain features of noisy speech trained for the determined acoustic environment, and applying a noise reduction mask on the received input audio segments according to the determination of the received audio segment is noisy speech

What is claimed is: 
     
         1 . A method of training a speech detection and speech enhancement unit of a Multipoint Conferencing Node (MCN), comprising:
 collecting, for each of a plurality of acoustic environments, a plurality of audio samples consisting of noise, clean speech and noisy speech;   labeling each of the plurality of samples with an acoustic environmental label corresponding to one of the plurality of acoustic environments;   labeling the plurality of audio samples consisting of clean speech with clean speech label;   extracting Time-Frequency (T-F) domain features from the plurality of audio samples consisting of noise, clean speech and noisy speech;   training, for each of the plurality of acoustic environments, one speech detection classifier by inputting the T-F domain features of the plurality of audio samples consisting of noise, clean speech and noisy speech, the acoustic environmental labels and the clean speech labels to a first deep neural network; and
 training, for each of the plurality of environments, one statistical generative model representing the probability distributions of the T-F domain features of noisy speech, by inputting the T-F domain features of the plurality of audio samples comprising clean speech, the T-F domain features of the plurality audio samples comprising noise and the plurality of audio samples comprising noisy speech. 
   
     
     
         2 . The method of  claim 1 , wherein the plurality of acoustic environments comprises meeting room with video conferencing endpoint, home office, and public space. 
     
     
         3 . The method of  claim 1 , wherein the statistical generative model is a Gaussian Mixture Model. 
     
     
         4 . A method of speech detection and speech enhancement in a speech detection and speech enhancement unit of Multipoint Conferencing Node (MCN), comprising:
 receiving input audio segments from at least one videoconferencing participants;   determining an acoustic environment based on auxiliary information of the at least one videoconferencing participant;   extracting Time-Frequency (T-F) domain features from the received input audio segments;   determining if each of the received input audio segments is speech by inputting the T-F domain features into a speech detection classifier trained for the determined acoustic environment;   determining, when one of the received input audio segments is speech, if the received audio segment is noisy speech by inputting the T-F domain features into a noise classifier using a statistical generative model representing the probability distributions of the T-F domain features of noisy speech trained for the determined acoustic environment; and   applying a noise reduction mask on the received input audio segments according to the determination of the received audio segment is noisy speech.   
     
     
         5 . The method of  claim 4 , wherein the auxiliary information of the at least one videoconferencing participant comprises at least one of a number of participants in a video image received from the at least one videoconferencing participant, and a specification of a videoconferencing endpoint received from the at least one videoconferencing participant. 
     
     
         6 . The method of  claim 4 , wherein the acoustic environment comprises meeting room with video conferencing endpoint, home office, and public space. 
     
     
         7 . The method of  claim 4 , wherein the speech detection and speech enhancement unit is trained according the method of  claim 1 . 
     
     
         8 . The method of  claim 4 , wherein the noise classifier is a Bayesian classifier. 
     
     
         9 . The method of  claim 4 , wherein the noise reduction mask is a composite noise reduction mask. 
     
     
         10 . The method of  claim 8 , wherein the composite noise reduction mask is based on an estimated binary mask (EBM) generated using the Bayesian classifier. 
     
     
         11 . The method of  claim 4 , further comprising updating the statistical generative model representing the probability distributions of the T-F domain features of noisy speech trained for the determined acoustic environment when the estimated probability of one the received input audio segments is noisy speech is close an estimated probability of the one received input audio segments is belonging clean speech. 
     
     
         12 . The method of  claim 9 , wherein the composite noise reduction mask is based on an estimated binary mask (EBM) generated using the Bayesian classifier.