Controlling loudness of speech in signals that contain speech and other types of audio material

Abstract

Mechanisms are known that allow receivers to control loudness of speech in broadcast signals but these mechanisms require an estimate of speech loudness be inserted into the signal. Disclosed techniques provide improved estimates of loudness. According to one implementation, an indication of the loudness of an audio signal containing speech and other types of audio material is obtained by classifying segments of audio information as either speech or non-speech. The loudness of the speech segments is estimated and this estimate is used to derive the indication of loudness. The indication of loudness maybe used to control audio signal levels so that variations in loudness of speech between different programs is reduced. A preferred method for classifying speech segments is described.

The invention claimed is: 
     
       1. A method for signal processing that comprises:
 receiving an audio signal; 
 extracting features of the audio signal; 
 analyzing one or more of the extracted features to perform a speech determination; 
 classifying segments within an interval of the audio signal as speech segments or non-speech segments based upon the speech determination, wherein each segment has a respective loudness, and the loudness or the speech segments is less than the loudness of one or more loud non-speech segments; 
 analyzing one or more of the extracted features of the audio signal to obtain an estimated loudness of the speech segments; and 
 providing an indication of the loudness of the interval of the audio signal by calculating control information from a weighted combination of the estimated loudness of the speech segments and the loudness of the non-speech segments in which the estimated loudness of the speech segments is weighted more heavily. 
 
     
     
       2. The method according to  claim 1  that comprises:
 controlling the loudness of the interval of the audio signal in response to the control information so as to reduce variations in the loudness of the speech segments, wherein the loudness of the portions of the audio signal represented by the one or more loud non-speech segments is increased when the loudness of the portions of the audio signal represented by the speech-segments is increased. 
 
     
     
       3. The method according to  claim 1  that comprises:
 assembling a representation of the audio signal and the control information into an output signal and transmitting the output signal. 
 
     
     
       4. A non-transitory computer-readable storage medium storing instructions for instructing processing circuitry to perform any one of the methods of  claim 1  through  3 . 
     
     
       5. An apparatus for signal processing that comprises:
 an input terminal that receives an input signal; 
 memory; and 
 processing circuitry coupled to the input terminal and the memory; wherein the processing circuitry performs any one of the methods of  claims 1  through  3 . 
 
     
     
       6. The method according to  claim 1  or  2  that obtains the estimated loudness of the speech segments by calculating average power of a frequency-weighted version of the audio signal represented by the speech segments. 
     
     
       7. The method according to  claim 1  or  2  that obtains the estimated loudness of the speech segments by applying a psychoacoustic model of loudness to the audio information. 
     
     
       8. The method according to  claim 1  or  2  that classifies segments by deriving from the extracted features a plurality of characteristics of the audio signal, weighting each characteristic by a respective measure of importance, and classifying the segments according to a combination of the weighted characteristics. 
     
     
       9. The method according to  claim 1  or  2  that controls the loudness of the interval of the audio signal by adjusting the loudness only during intervals of the audio signal having a measure of audio energy less than a threshold. 
     
     
       10. The method according to  claim 1  or  2  wherein
 the weighting of the loudness of the non-speech segments in the weighted combination is zero. 
 
     
     
       11. The method according to  claim 1  or  2  that comprises
 analyzing one or more of the extracted features of the audio signal to obtain an estimate of the loudness of one or more non-speech segments. 
 
     
     
       12. The method according to  claim 1  or  2  that comprises:
 providing a speech measure that indicates a degree to which the audio signal represented by a respective segment has characteristics of speech; and 
 providing the indication of loudness by calculating the control information in response to the estimated loudness of respective segments according to the speech measures of the respective segments. 
 
     
     
       13. The method according to  claim 1  or  2  that comprises calculating the control information in response to the estimated loudness of respective segments according to time order of the segments. 
     
     
       14. The method according to  claim 1  or  2  that comprises
 adapting lengths of the segments in response to characteristics of the audio signal. 
 
     
     
       15. A non-transitory computer-readable storage medium storing instructions for instructing processing circuitry to perform the method of  claim 6 . 
     
     
       16. A non-transitory computer-readable storage medium storing instructions for instructing processing circuitry to perform the method of  claim 7 . 
     
     
       17. A non-transitory computer-readable storage medium storing instructions for instructing processing circuitry to perform the method of  claim 8 . 
     
     
       18. A non-transitory computer-readable storage medium storing instructions for instructing processing circuitry to perform the method of  claim 9 . 
     
     
       19. A non-transitory computer-readable storage medium storing instructions for instructing processing circuitry to perform the method of  claim 10 . 
     
     
       20. A non-transitory computer-readable storage medium storing instructions for instructing processing circuitry to perform the method of  claim 11 . 
     
     
       21. A non-transitory computer-readable storage medium storing instructions for instructing processing circuitry to perform the method of  claim 12 . 
     
     
       22. A non-transitory computer-readable storage medium storing instructions for instructing processing circuitry to perform the method of  claim 13 . 
     
     
       23. A non-transitory computer-readable storage medium storing instructions for instructing processing circuitry to perform the method of  claim 14 . 
     
     
       24. An apparatus for signal processing that comprises:
 an input terminal that receives an input signal; 
 memory; and 
 processing circuitry coupled to the input terminal and the memory; wherein the processing circuitry performs the method of  claim 6 . 
 
     
     
       25. An apparatus for signal processing that comprises:
 an input terminal that receives an input signal; 
 memory; and 
 processing circuitry coupled to the input terminal and the memory; wherein the processing circuitry performs the method of  claim 7 . 
 
     
     
       26. An apparatus for signal processing that comprises:
 an input terminal that receives an input signal; 
 memory; and 
 processing circuitry coupled to the input terminal and the memory; wherein the processing circuitry performs the method of  claim 8 . 
 
     
     
       27. An apparatus for signal processing that comprises:
 an input terminal that receives an input signal; 
 memory; and 
 processing circuitry coupled to the input terminal and the memory; wherein the processing circuitry performs the method of  claim 9 . 
 
     
     
       28. An apparatus for signal processing that comprises:
 an input terminal that receives an input signal; 
 memory; and 
 processing circuitry coupled to the input terminal and the memory; wherein the processing circuitry performs the method of  claim 10 . 
 
     
     
       29. An apparatus for signal processing that comprises:
 an input terminal that receives an input signal; 
 memory; and 
 processing circuitry coupled to the input terminal and the memory; wherein the processing circuitry performs the method of  claim 11 . 
 
     
     
       30. An apparatus for signal processing that comprises:
 an input terminal that receives an input signal; 
 memory; and 
 processing circuitry coupled to the input terminal and the memory; wherein the processing circuitry performs the method of  claim 12 . 
 
     
     
       31. An apparatus for signal processing that comprises:
 an input terminal that receives an input signal; 
 memory; and 
 processing circuitry coupled to the input terminal and the memory; wherein the processing circuitry performs the method of  claim 13 . 
 
     
     
       32. An apparatus for signal processing that comprises:
 an input terminal that receives an input signal; 
 memory; and 
 processing circuitry coupled to the input terminal and the memory; wherein the processing circuitry performs the method of  claim 14 . 
 
     
     
       33. A method for signal processing that comprises:
 receiving an input audio signal; 
 extracting features of the input audio signal, the extracted features representing an interval of the input of audio signal; 
 analyzing the extracted features to perform a speech determination; 
 classifying the interval of the audio signal as speech or non-speech based upon the speech determination, wherein each interval has a respective loudness and the a loudness of the interval classified as speech is less than the a loudness of one or more other segments classified as non-speech; 
 analyzing the extracted features of the interval classified as speech to obtain an estimated loudness of the interval classified as speech; 
 calculating a loudness control parameter, the loudness control parameter being proportional to the difference between the estimated loudness of intervals classified as speechadjusting a loudness of the interval classified as speech, the adjustment being determined by a loudness control parameter and the estimated loudness; and 
 adjusting an estimated a loudness of the one or more other intervals classified as non-speech, the adjustment being proportional to the calculated loudness control parameter. 
 
     
     
       34. A non-transitory computer-readable storage medium storing instructions for instructing processing circuitry to perform the method of  claim 33 . 
     
     
       35. An apparatus for signal processing that comprises:
 an input terminal that receives an input signal; 
 memory; and 
 processing circuitry coupled to the input terminal and the memory; wherein the processing circuitry performs the method of  claim 33 . 
 
     
     
       36. A method for signal processing that comprises:
 receiving an input audio signal;   extracting features for a first segment, second segment, and third segment of the input audio signal;   analyzing the extracted features to classify the first and third segments as speech segments;   analyzing the extracted features to classify the second segment as a non-speech segment, the second segment being louder than at least one of the first and third segments;   analyzing the first segment to determine a first estimated loudness;   analyzing the third segment to determine a second estimated loudness; and   adjusting, a loudness control parameter, based at least upon the first estimated loudness and the third estimated loudness, for the third segment to a reduce variation of speech loudness between the first and third segments despite the presence of the second segment.   
     
     
       37. The method of claim 36 wherein the method is performed by one or more devices comprising a processor. 
     
     
       38. The method of claim 36 wherein the second segment comprises a sound effect. 
     
     
       39. The method of claim 36 wherein the second segment comprises music. 
     
     
       40. The method of 36 wherein the speech loudness between the first and third segments after the adjusting is a constant level. 
     
     
       41. The method of 36 further comprising adjusting a loudness of the second segment as determined by the adjusting of loudness of the third segment. 
     
     
       42. The method of claim 36 wherein a first televised program includes the first segment, and a second televised program includes the third segment. 
     
     
       43. The method of claim 36 wherein a first televised program includes the first segment, and a commercial announcement includes the second segment. 
     
     
       44. The method of claim 36 wherein the first segment is received on a first channel, and the second segment is received on a second channel. 
     
     
       45. A method for signal processing, the method comprising:
 receiving an input audio signal;   extracting features of an interval of the input audio signal for a first segment and a second segment of the input audio signal;   analyzing the extracted features to classify the first and second segments as speech segments;   analyzing the first segment to determine a first estimated loudness;   analyzing the second segment to determine a second estimated loudness; and   adjusting a loudness of the second segment to reduce variation of speech loudness between the first and second segments using a loudness control parameter that is based, at least in part, on the first estimated loudness and the second estimated loudness.   
     
     
       46. The method of claim 45 wherein the method is performed by one or more devices comprising a processor. 
     
     
       47. The method of claim 45 wherein a broadcast program includes the first segment and a commercial announcement includes the second segment. 
     
     
       48. The method of claim 45 wherein the first segment is received on a first channel of the input audio signal, and the second segment is received on a second channel of the input audio signal. 
     
     
       49. A method for signal processing, the method comprising:
 receiving an input audio signal;   extracting features of an interval of the input audio signal for a first segment and a second segment of the input audio signal;   analyzing the extracted features to classify the first and second segments as speech segments;   analyzing the first segment to determine a first estimated loudness;   analyzing the second segment to determine a second estimated loudness; and   adjusting a loudness of the first segment to reduce variation of speech loudness between the first and second segments based at least upon the first estimated loudness and the second estimated loudness.   
     
     
       50. The method of claim 49 wherein the method is performed by one or more devices comprising a processor. 
     
     
       51. The method of claim 49 wherein a broadcast program includes the first segment and a commercial announcement includes the second segment. 
     
     
       52. The method of claim 49 wherein the first segment is received on a first channel of the input audio signal, and the second segment is received on a second channel of the input audio signal.