Unified audio suppression model

Abstract

Examples herein provide an approach to enhance an audio mixture of a teleconference application by switching between noise suppression modes using a single model. Specifically, a machine learning (ML) model may be configured to, in response to receiving an audio mixture representation as input, suppress either a background noise of the audio mixture or suppress all noise of the audio mixture except a user's voice. In some examples, the ML model may be trained on speech and background noise training data during a training phase. In addition, the ML model may be trained on a user's voice during an enrollment phase. In addition, during an inference phase, the ML model may enhance the audio mixture by suppressing a portion of the audio mixture.

What is claimed is: 
     
         1 . A system for enhancing teleconference application audio, the system comprising:
 memory that stores computer-executable instructions; and   a processor in communication with the memory, wherein the computer-executable instructions, when executed by the processor, cause the processor to:
 obtain a voice sample of a user; 
 map the voice sample to a user identifier; 
 receive an audio mixture detected by an audio sensor; 
 receive a selection via a teleconference application that identifies a portion of the audio mixture to suppress; 
 modify a representation of the audio mixture to include a flag that corresponds to the selection; and 
 apply the modified representation of the audio mixture as an input into a machine learning model, wherein application of the modified representation of the audio mixture as the input to the machine learning model causes the machine learning model to one of:
 suppress a background noise of the audio mixture, or 
 suppress all noise of the audio mixture except a voice identified by the user identifier. 
 
   
     
     
         2 . The system of  claim 1 , wherein the modified representation of the audio mixture includes the user identifier, the audio mixture, and the flag. 
     
     
         3 . The system of  claim 1 , wherein the flag is a binary bit indicating whether the selection corresponds to background noise suppression or all noise suppression except the voice identified by the user identifier. 
     
     
         4 . The system of  claim 1 , wherein suppressing the background noise of the audio mixture comprises preserving a second voice of a second user from being suppressed. 
     
     
         5 . The system of  claim 1 , wherein the machine learning model is trained on combined training data that comprises a first training data item,
 wherein the first training data item includes a combination of a first type of clean speech data and a first type of background noise data, and   wherein the first type of clean speech data is identified as a target output.   
     
     
         6 . The system of  claim 1 , wherein the machine learning model is trained on the voice sample of the user during an enrollment phase. 
     
     
         7 . The system of  claim 1 , wherein the computer-executable instructions, when executed, further cause the processor to:
 receive, during a teleconference session in which the selection is received, a second selection via the teleconference application that identifies a second portion of the audio mixture to suppress that is different than the portion of the audio mixture; and   cause the second portion of the audio mixture to be suppressed.   
     
     
         8 . A method for enhancing audio of a communication application, the method comprising:
 memory that stores computer-executable instructions; and
 obtaining a voice sample of a user; 
 mapping the voice sample to a user identifier; 
 receiving an audio mixture detected by an audio sensor; 
 receiving a selection via a communication application that identifies a portion of the audio mixture to enhance; 
 modifying a representation of the audio mixture to include a flag that corresponds to the selection; and 
 applying the modified representation of the audio mixture as an input into a machine learning model, wherein application of the modified representation of the audio mixture as the input to the machine learning model causes the machine learning model to enhance a portion of the audio mixture corresponding to the selection. 
   
     
     
         9 . The method of  claim 8 , wherein the modified representation of the audio mixture includes the user identifier, the audio mixture, and the flag. 
     
     
         10 . The method of  claim 8 , wherein the flag is a binary bit indicating whether the selection corresponds to background noise suppression or all noise suppression except the voice identified by the user identifier. 
     
     
         11 . The method of  claim 8 , wherein the portion of the audio mixture includes a background noise of the audio mixture or all noise audio mixture except a voice identified by the user identifier. 
     
     
         12 . The method of  claim 8 , wherein the machine learning model is further caused to suppress a background noise of the audio mixture and preserve a second voice of a second user from being suppressed. 
     
     
         13 . The method of  claim 8 , wherein the machine learning model is trained on the voice sample of the user during an enrollment phase. 
     
     
         14 . A non-transitory, computer-readable medium comprising computer-executable instructions for enhancing audio of a communication application, wherein the computer-executable instructions, when executed by a computer system, cause the computer system to:
 receive an audio mixture detected by an audio sensor;   receive a selection via the communication application that identifies a portion of the audio mixture to enhance;   modify a representation of the audio mixture to include a flag that corresponds to the selection; and   apply the modified representation of the audio mixture as an input into a machine learning model, wherein application of the modified representation of the audio mixture as the input to the machine learning model causes the machine learning model to enhance a portion of the audio mixture corresponding to the selection.   
     
     
         15 . The non-transitory, computer-readable medium of  claim 14 , wherein the modified representation of the audio mixture includes a user identifier, the audio mixture, and the flag. 
     
     
         16 . The non-transitory, computer-readable medium of  claim 14 , wherein the flag is a binary bit indicating whether the selection corresponds to background noise suppression or all noise suppression except a voice identified by a user identifier. 
     
     
         17 . The non-transitory, computer-readable medium of  claim 14 , wherein the machine learning model is trained on a voice sample of a user during an enrollment phase. 
     
     
         18 . The non-transitory, computer-readable medium of  claim 15 , wherein the portion of the audio mixture includes a background noise of the audio mixture or all noise audio mixture except a voice identified by the user identifier. 
     
     
         19 . The non-transitory, computer-readable medium of  claim 14 , wherein the computer-executable instructions, when executed, further cause the computer system to suppress a second voice of a second user. 
     
     
         20 . The non-transitory, computer-readable medium of  claim 14 , wherein the computer-executable instructions, when executed, further cause the computer system to preserve a second voice of a second user from being suppressed.