Three-dimensional audio systems

Abstract

A three-dimensional sound generation system includes one or more processors of a computing device, including capability to receive sound tracks, each of the sound tracks comprising one or more sound sources, each of the one or more sound sources corresponding to one or more respective sound categories, receive or determine a first configuration in a three-dimensional space, the first configuration comprising a listener position and a computing device location relative to the listener position, determine a second configuration comprising a change to at least one of the listener location or the computing device location relative to the listener position, generate, using the one or more sound tracks and the second configuration, one or more channels of sound signals, and provide the one or more channels of sound signals to drive one or more sound generation devices to generate a three-dimensional sound field.

1 . A three-dimensional sound generation system, comprising:
 one or more processors of a computing device, communicatively coupled to or embedded with a user interface device, the three-dimensional sound generation system having capability to:
 receive one or more sound tracks, each of the one or more sound tracks comprising one or more sound sources, each of the one or more sound sources corresponding to one or more respective sound categories; 
 receive or determine a first configuration in a three-dimensional space, the first configuration comprising a listener position and a computing device location relative to the listener position; 
 determine a second configuration comprising a change to at least one of the listener location or the computing device location relative to the listener position; 
 responsive to determining at least one of the first configuration or the second configuration, generate, using the one or more sound tracks and the second configuration, one or more channels of sound signals; and 
 provide the one or more channels of sound signals to drive one or more sound generation devices to generate a three-dimensional sound field that renders a sound effect as if the listener at the listener position according to the at least one of the first configuration or the second configuration perceives the one or more sound sources originating from the computing device location in the three-dimensional space. 
   
     
     
         2 . The three-dimensional sound generation system of  claim 1 , wherein the computing device is one of a smartphone, a smart speaker, a soundbar, a television set, a game console, a home/theater sound system, a computer, a tablet computer, an automobile vehicle, a headset, earphones, headphones, earbuds, headsets, a helmet, or a cloud server, and wherein the one or more sound generation devices comprise one or more earphones, headphones, earbuds, headsets, or loudspeakers, and wherein the sound category comprises one or more of a voice sound, a vocal sound, an instrument sound, a car sound, a helicopter sound, an airplane sound, a vehicle sound, a gunshot sound, a footstep sound, an explosion sound, a sound in a movie, a sound in a game, or an environmental sound. 
     
     
         3 . The three-dimensional sound generation system of  claim 1 , wherein the three-dimensional sound generation system has further capability to:
 responsive to receiving or determining the first configuration, generate, using the one or more sound tracks and the first configuration, the one or more channels of sound signals; and   provide the one or more channels of sound signals to drive the one or more sound generation devices to generate the three-dimensional sound field that renders the sound effect as if the listener at the listener position according to the first configuration perceives the one or more sound sources originating from the computing device location in the three-dimensional space.   
     
     
         4 . The three-dimensional sound generation system of  claim 1 , wherein the three-dimensional sound generation system has further capability to:
 responsive to receiving or determining the first configuration, present in a configuration environment, a representation representing a listener at the listener location;   determine a second configuration comprising a change to the listener location; and   present, in the configuration environment, the representation representing the listener at the changed listener location.   
     
     
         5 . The three-dimensional sound generation system of  claim 1 , wherein the three-dimensional sound generation system has further capability to:
 determine, by using one or more sensors, at least one of a relative location or a movement between the computing device and the listener;   responsive to determining at least one of the relative location or the movement between the computing device and the listener, determine the second configuration.   
     
     
         6 . The three-dimensional sound generation system of  claim 1 , wherein the three-dimensional sound generation system has further capability to:
 responsive to determining the second configuration, determine a plurality of head related transfer function (HRTF) filters based on the second configuration comprising the listener location and the computing device location relative to the listener position; and   generate, using the one or more sound tracks and the plurality of HRTF filters, the one or more channels of sound signals to render the three-dimensional sound effect.   
     
     
         7 . The three-dimensional sound generation system of  claim 1 , wherein the three-dimensional sound generation system has the capability performed by the one or more processors of the computing device. 
     
     
         8 . The three-dimensional sound generation system of  claim 1 , wherein the three-dimensional sound generation system has the capability performed by one or more processors embedded in the sound generation device. 
     
     
         9 . A method comprising:
 receiving one or more sound tracks, each of the sound tracks comprising one or more sound sources, each of the one or more sound sources corresponding to one or more respective sound categories;   receiving or determining a first configuration in a three-dimensional space, the first configuration comprising a listener position and a computing device location relative to the listener position;   determining a second configuration comprising a change to at least one of the listener location or the computing device location relative to the listener position;   responsive to determining the second configuration, generating, using the one or more sound tracks and the second configuration, one or more channels of sound signals; and   providing the one or more channels of sound signals to drive one or more sound generation devices to generate a three-dimensional sound field that renders a sound effect as if the listener at the listener position according to the second configuration perceives the one or more sound sources originating from the computing device location in the three-dimensional space.   
     
     
         10 . A three-dimensional sound generation system, comprising one or more processors, the three-dimensional sound generation system having capability to:
 receive a sound from one or more sound sources, wherein each of the one or more sound sources corresponds to one or more respective sound categories;   receive or determine a first configuration in a three-dimensional space, the first configuration comprising a listener position and one or more sound source locations representing a listener at the listener location in the three-dimensional space and the one or more sound sources at the one or more sound source locations;   determine a second configuration comprising a change to at least one of the listener location or the one or more sound source locations in the first configuration;   separate the sound into one or more sound tracks, wherein each of the one or more sound track representing one or more sound categories from the one or more locations in the three-dimensional space;   generate, using the separated one or more sound tracks and at least one of the first or the second configuration, one or more channels of sound signals; and   provide the one or more channels of sound signals to drive one or more sound generation devices to generate a three-dimensional sound field that renders a sound effect as if the listener at the listener position according to the second configuration listens to the one or more sound sources at the one or more sound source locations in the three-dimensional space.   
     
     
         11 . The three-dimensional sound generation system of  claim 10 , wherein the computing device is one of a smartphone, a smart speaker, a soundbar, a television set, a game console, a home/theater sound system, a computer, a tablet computer, an automobile vehicle, a headset, earphones, headphones, earbuds, headsets, a loudspeaker, a helmet, or a cloud server, and wherein the one or more sound generation devices comprise one or more earphones, headphones, earbuds, headsets, or loudspeakers, and wherein the sound category comprises one or more of a voice sound, a vocal sound, an instrument sound, a car sound, a helicopter sound, an airplane sound, a vehicle sound, a gunshot sound, a footstep sound, an explosion sound, a sound in a movie, a sound in a game, an environmental sound, a natural sound, an artificial sound, or a computer-generated sound. 
     
     
         12 . The three-dimensional sound generation system of  claim 10 , wherein the three-dimensional sound generation system has further capability to:
 responsive to receiving or determining the first configuration, present in a configuration environment, representations representing a listener at the listener location and one or more sound sources at the one or more sound source locations; and   responsive to a user interaction using the configuration environment with the representations representing the listener at the listener location and the one or more sound sources at the one or more sound source locations, determine the second configuration comprising the change to at least one of the listener location or the one or more sound source locations in the first configuration.   
     
     
         13 . The three-dimensional sound generation system of  claim 12 , wherein the configuration environment comprises one or more graphic user interfaces, wherein the representations are graphic icons in the graphic user interface, and wherein the user interaction comprises an action directed at the graphic representations in the graphic user interface, wherein the second configuration comprises a virtual change to the one or more sound source locations in the first configuration by the user using the one or more graphic user interface. 
     
     
         14 . The three-dimensional sound generation system of  claim 10 , wherein the three-dimensional sound generation system has further capability to:
 responsive to receiving or determining the second configuration, select a plurality of head related transfer function (HRTF) filters based on the second configuration comprising the listener location and the one or more sound source locations; and   generating, using the separated one or more sound tracks, the second configuration, and the plurality of HRTF filters, one or more channels of sound signals; and   providing the one or more channels of sound signals to drive one or more sound generation devices to generate a three-dimensional sound field that renders a sound effect as if the listener at the listener position according to the second configuration listens to the one or more sound sources at the one or more sound source locations in the three-dimensional space.   
     
     
         15 . A method comprising:
 receiving a sound generated from one or more sound sources, wherein each of the one or more sound sources corresponds to one or more respective sound categories;   determining a first configuration in a three-dimensional space, the first configuration comprising a listener position and one or more sound source locations representing a listener at the listener location in the three-dimensional space and the one or more sound sources at the one or more sound source locations;   determining a second configuration comprising a change to at least one of the listener location or the one or more sound source locations in the first configuration;   separating the sound into one or more sound tracks, wherein each of the one or more sound track representing one or more sound categories from the one or more locations in the three-dimensional space;   generating, using the separated one or more sound tracks and the second configuration, one or more channels of sound signals; and   providing the one or more channels of sound signals to drive one or more sound generation devices to generate a three-dimensional sound field that renders a sound effect as if the listener at the listener position according to the second configuration listens to the one or more sound sources at the one or more sound source locations in the three-dimensional space.   
     
     
         16 . A three-dimensional sound generation system, comprising one or more processors, communicatively coupled to one or more sound generation devices, the three-dimensional sound generation system having capability to:
 receive a sound that is composed of the one or more sound sources, each of the one or more sound sources corresponding to a respective sound category;   obtain a configuration of a three-dimensional space comprising one or more listener positions and one or more the sound source locations;   select one or more head related transfer function (HRTF) filters from a predetermined array of HRTF filters based on the relative positions of the sound source locations and the one or more sound listeners in the three-dimensional space;   generate, using the select one or more HRTF filters and the relative positions of the sound source locations and the one or more sound generation devices in the three-dimensional space, one or more channels of sound signals; and   generate a three-dimensional sound field that renders a sound effect as if one or more listeners at the one or more listener positions listen to the one or more sound sources at the one or more sound source locations in the three-dimensional space.   
     
     
         17 . The three-dimensional sound generation system of  claim 16 , wherein the three-dimensional sound generation system has further capability to:
 dynamically update the selection of the one or more HRTF filters selected from the predetermined array of HRTF filters based on an update to the relative positions of the sound source locations and the one or more sound generation devices in the three-dimensional space.   
     
     
         18 . The three-dimensional sound generation system of  claim 16 , wherein the one or more sound generation devices comprise one or more earphones, headphones, earbuds, headsets, or loudspeakers, and wherein the sound category comprises one or more of a voice sound, a vocal sound, an instrument sound, a car sound, a helicopter sound, an airplane sound, a vehicle sound, a gunshot sound, a footstep sound, an explosion sound, a sound in a movie, a sound in a game, or an environmental sound. 
     
     
         19 . The three-dimensional sound generation system of  claim 16 , wherein the three-dimensional sound generation system has further capability to:
 responsive to receiving or determining the first configuration, present in a configuration environment, representations representing one or more sound sources at the one or more sound source locations and one or more sound generation devices at the one or more sound generation locations; and   responsive to a change in the configuration environment with the representations representing the one or more sound sources at the one or more sound source locations and one or more sound generation devices at the one or more sound generation locations, determine the second configuration comprising the change to at least one of the one or more sound source locations or the one or more sound generation devices at the one or more sound generation device locations in the first configuration.   
     
     
         20 . The three-dimensional sound generation system of  claim 19 , wherein the configuration environment comprises a graphic user interface, one or more files, or one or more computer programs, wherein the representations are graphic representations in the graphic user interface, and wherein the change comprises an action directed at the graphic representations in the graphic user interface. 
     
     
         21 . The three-dimensional sound generation system of  claim 16 , wherein the predetermined array of HRTF filters comprise HRTF filters defined on nodes of a mesh of grids. 
     
     
         22 . A method comprising:
 receiving a sound that is composed of the one or more sound sources, each of the one or more sound sources corresponding to a respective sound category;   obtaining a configuration of a three-dimensional space comprising one or more listener positions and one or more the sound source locations;   selecting one or more head related transfer function (HRTF) filters from a predetermined array of HRTF filters based on the relative positions of the sound source locations and the one or more sound listeners in the three-dimensional space;   generating, using the select one or more HRTF filters and the relative positions of the sound source locations and the one or more sound generation devices in the three-dimensional space, one or more channels of sound signals; and   generating a three-dimensional sound field that renders a sound effect as if one or more listeners at the one or more listener positions listen to the one or more sound sources at the one or more sound source locations in the three-dimensional space.