Spatial audio representation and rendering

Abstract

An apparatus including circuitry configured to: receive a spatial audio signal, the spatial audio signal including at least one audio signal and spatial metadata associated; generate at least one decorrelated audio signal; determine at least one control parameter, wherein the at least one control parameter is at least based on at least one target further property of the at least two output audio signals and at least one of: the spatial metadata and at least one property determined based on the at least one audio signal; and generate the at least two output audio signals based on the spatial audio signal and at least one decorrelated audio signal, wherein the amount of the at least one decorrelated audio signal within at least two output audio signals is controlled based on the at least one control parameter.

The invention claimed is: 
     
       1. An apparatus comprising:
 at least one processor; and 
 at least one memory storing instructions that, when executed with the at least one processor, cause the apparatus to:
 receive a spatial audio signal, the spatial audio signal comprising at least one audio signal and spatial metadata associated with the at least one audio signal; 
 generate at least one decorrelated audio signal based on the at least one audio signal; 
 determine at least one control parameter configured to control an amount of the at least one decorrelated audio signal within at least two output audio signals for spatial audio reproduction, wherein the at least one control parameter is at least based on at least one target further property of the at least two output audio signals and at least one of: the spatial metadata or at least one property determined based on the at least one audio signal; and 
 generate the at least two output audio signals for the spatial audio reproduction based on the spatial audio signal and the at least one decorrelated audio signal, wherein the amount of the at least one decorrelated audio signal within at least two output audio signals is controlled based on the at least one control parameter; 
 wherein the at least one target further property comprises an audio type or covariance. 
 
 
     
     
       2. The apparatus as claimed in  claim 1 , wherein the at least one control parameter comprises at least one of:
 at least one processing gain applied to at least one of the at least one decorrelated audio signal or the at least one audio signal being decorrelated; 
 at least one mixing matrix configured to control a mixing of the at least one decorrelated audio signal and the at least one audio signal; 
 at least one mixing matrix and at least one residual mixing matrix, the at least one mixing matrix and the at least one residual mixing matrix configured to control a mixing of the at least one decorrelated audio signal and the at least one audio signal; or 
 at least one covariance matrix configured to control a generation of at least one mixing matrix and/or at least one residual mixing matrix, the at least one mixing matrix and/or the at least one residual mixing matrix configured to control a mixing of the at least one decorrelated audio signal and/or the at least one audio signal. 
 
     
     
       3. The apparatus as claimed in  claim 1 , wherein the instructions, when executed with the at least one processor, cause the apparatus to:
 determine at least one further property based on the at least one audio signal; 
 determine the at least one target further property of the at least two output audio signals; 
 determine at least one first control parameter based on the at least one further property based on the at least one audio signal and the at least one target further property of the at least two output audio signals; and 
 determine at least one second control parameter or modify the at least one first control parameter based on at least one of: the spatial metadata or the at least one property determined based on the at least one audio signal. 
 
     
     
       4. The apparatus as claimed in  claim 3 , wherein the instructions, when executed with the at least one processor, cause the apparatus to at least one of:
 mix the at least one audio signal and the at least one decorrelated audio signal based on the at least one first control parameter and the at least one second control parameter or the at least one modified first control parameter; or 
 output the at least two output audio signals for the spatial audio reproduction. 
 
     
     
       5. The apparatus as claimed in  claim 3 , wherein the determined the at least one second control parameter or the modified at least one first control parameter is based on at least one direct-to-total energy ratio parameter within the spatial metadata. 
     
     
       6. The apparatus as claimed in  claim 3 , wherein the at least one further property based on the at least one audio signal is a covariance property, and the at least one target further property of the at least two output audio signals is a target covariance property of the at least two output audio signals. 
     
     
       7. The apparatus as claimed in  claim 6 , wherein the instructions, when executed with the at least one processor, cause the apparatus to at least one of:
 determine a residual covariance property based on the covariance property and the target covariance property of the at least two output audio signals; or 
 process the residual covariance property based on the spatial metadata associated with the at least one audio signal. 
 
     
     
       8. The apparatus as claimed in  claim 7 , wherein the instructions, when executed with the at least one processor, cause the apparatus to:
 attenuate the residual covariance property when the spatial metadata indicates that the at least one audio signal is directional; and 
 pass the residual covariance property unprocessed when the spatial metadata indicates that the at least one audio signal is ambient. 
 
     
     
       9. The apparatus as claimed in  claim 6 , wherein the instructions, when executed with the at least one processor, cause the apparatus to:
 generate an overall energy estimate based on the covariance property; 
 determine head related transfer function data based on a direction parameter from the spatial metadata associated with the at least one audio signal; and 
 determine the target covariance property of the at least two output audio signals further based on the head related transfer function data and the overall energy estimate. 
 
     
     
       10. The apparatus as claimed in  claim 1 , wherein the determined at least one property is based on the at least one audio signal, the at least one property is the audio type, and the instructions, when executed with the at least one processor, cause the apparatus to:
 determine whether the audio type is a determined audio type; and 
 determine the at least one control parameter based on the audio type being the determined audio type. 
 
     
     
       11. The apparatus as claimed in  claim 10 , wherein the determined audio type is speech. 
     
     
       12. The apparatus as claimed in  claim 1 , wherein the at least one audio signal comprises transport audio signals generated with an encoder. 
     
     
       13. A method for an apparatus, the method comprising:
 receiving a spatial audio signal, the spatial audio signal comprising at least one audio signal and spatial metadata associated with the at least one audio signal; 
 generating at least one decorrelated audio signal based on the at least one audio signal; 
 determining at least one control parameter configured to control an amount of the at least one decorrelated audio signal within at least two output audio signals for spatial audio reproduction, wherein the at least one control parameter is at least based on at least one target further property of the at least two output audio signals and at least one of: the spatial metadata or at least one property determined based on the at least one audio signal; and 
 generating the at least two output audio signals for the spatial audio reproduction based on the spatial audio signal and the at least one decorrelated audio signal, wherein the amount of the at least one decorrelated audio signal within at least two output audio signals is controlled based on the at least one control parameter; 
 wherein the at least one target further property comprises an audio type or covariance. 
 
     
     
       14. The method as claimed in  claim 13 , wherein the at least one control parameter comprises at least one of:
 at least one processing gain applied to at least one of the at least one decorrelated audio signal or the at least one audio signal being decorrelated; 
 at least one mixing matrix configured to control a mixing of the at least one decorrelated audio signal and the at least one audio signal; 
 at least one mixing matrix and at least one residual mixing matrix, the at least one mixing matrix and the at least one residual mixing matrix configured to control a mixing of the at least one decorrelated audio signal and the at least one audio signal; or 
 at least one covariance matrix configured to control a generation of at least one mixing matrix and/or at least one residual mixing matrix, the at least one mixing matrix and/or the at least one residual mixing matrix configured to control a mixing of the at least one decorrelated audio signal and/or the at least one audio signal. 
 
     
     
       15. The method as claimed in  claim 13 , wherein determining the at least one control parameter further comprises:
 determining at least one further property based on the at least one audio signal; 
 determining the at least one target further property of the at least two output audio signals; 
 determining at least one first control parameter based on the at least one further property based on the at least one audio signal and the at least one target further property of the at least two output audio signals; and 
 determining at least one second control parameter or modify the at least one first control parameter based on at least one of: the spatial metadata or the at least one property determined based on the at least one audio signal. 
 
     
     
       16. The method as claimed in  claim 15 , wherein generating the at least two output audio signals for the spatial audio reproduction further comprises:
 mixing the at least one audio signal and the at least one decorrelated audio signal based on the at least one first control parameter and the at least one second control parameter or the at least one modified first control parameter; or 
 outputting the at least two output audio signals for the spatial audio reproduction. 
 
     
     
       17. The method as claimed in any of  claim 15 , wherein determining the at least one second control parameter or the modified at least one first control parameter is based on at least one direct-to-total energy ratio parameter within the spatial metadata. 
     
     
       18. The method as claimed in  claim 15 , wherein the at least one further property based on the at least one audio signal is a covariance property, and the at least one target further property of the at least two output audio signals is a target covariance property of the at least two output audio signals. 
     
     
       19. The method as claimed in  claim 18 , wherein determining the target covariance property of the at least two output audio signals comprises:
 generating an overall energy estimate based on the covariance property; 
 determining head related transfer function data based on a direction parameter from the spatial metadata associated with the at least one audio signal; and 
 determining the target covariance property of the at least two output audio signals further based on the head related transfer function data and the overall energy estimate. 
 
     
     
       20. The method as claimed in  claim 17 , wherein determining the at least one second control parameter or modifying the at least one first control parameter comprises at least one of:
 determining a residual covariance property based on the covariance property and the target covariance property of the at least two output audio signals; or 
 processing the residual covariance property based on the spatial metadata associated with the at least one audio signal, wherein processing the residual covariance property based on the spatial metadata associated with the at least one audio signal comprises:
 attenuating the residual covariance property when the spatial metadata indicates that the at least one audio signal is directional; and 
 passing the residual covariance property unprocessed when the spatial metadata indicates that the at least one audio signal is fully ambient.