Enhanced Immersive Soundscapes Production

Abstract

An immersive audio-visual system (and a method) for creating an enhanced interactive and immersive audio-visual environment is disclosed. The immersive audio-visual environment enables participants to enjoy true interactive, immersive audio-visual reality experience in a variety of applications. The immersive audio-visual system comprises an immersive video system, an immersive audio system and an immersive audio-visual production system. The video system creates immersive stereoscopic videos that mix live videos, computer generated graphic images and human interactions with the system. The immersive audio system creates immersive sounds with each sound resource positioned correct with respect to the position of an associated participant in a video scene. The immersive audio-video production system produces an enhanced immersive audio and videos based on the generated immersive stereoscopic videos and immersive sounds. A variety of applications are enabled by the immersive audio-visual production including casino-type interactive gaming system and training system.

1 . A computer method for determining location of a sound source for an immersive audio-visual production system having one or more of cameras and microphones, the method comprising:
 recording sound on multiple sound tracks, each sound track being associated with one of the microphones;   collecting sound source information from the multiple sound tracks;   analyzing the collected sound source information; and   determining location of the sound source.   
   
   
       2 . The method of  claim 1 , further comprising generating a first sound texture map based on the determined location of the sound source. 
   
   
       3 . The method of  claim 1 , further comprising constructing a three-dimensional video model using the cameras. 
   
   
       4 . The method of  claim 3 , wherein the three-dimensional video model contains one or more sound sources. 
   
   
       5 . The method of  claim 1 , wherein the sound source information from a sound track of the multiple sound tracks comprises one or more sound waves from the microphone associated with the sound track, a sound wave containing information about a sound source. 
   
   
       6 . The method of  claim 5 , wherein the information of a sound wave includes at least one of a group of distance between a sound source and the microphone, latency of the sound wave, delay of the sound wave and phase shift of the sound wave. 
   
   
       7 . The method of  claim 1 , further comprising reconciling the three-dimensional video model with the location of the sound source. 
   
   
       8 . The method of  claim 1 , further comprising:
 adding sounds from sound sources not contained in the three-dimensional video model; and   determining location of the added sound source;   
   
   
       9 . The method of  claim 8 , further comprising:
 adding the location of the added sound source to the first sound texture map to generate a composite sound texture map.   
   
   
       10 . A computer system for determining location of a sound source for an immersive audio-visual production, having one or more of cameras and microphones, the system comprising:
 a recording module configured to record sound on multiple sound tracks, each sound track being associated with one of the microphones; and   an immersive sound processing module configured to:
 collect sound source information from the multiple sound tracks; 
 analyze the collected sound source information; and 
 determined location of the sound source. 
   
   
   
       11 . The system of  claim 10 , wherein the immersive sound processing module is further configured to generate a first sound texture map based on the determined location of the sound source. 
   
   
       12 . The system of  claim 10 , further comprising an immersive video module configured to construct a three-dimensional video model using the the cameras. 
   
   
       13 . The system of  claim 12 , wherein the three-dimensional video model contains one or more sound sources. 
   
   
       14 . The system of  claim 10 , wherein the sound source information from a sound track of the multiple sound tracks comprises one or more sound waves from the microphone associated with the sound track, a sound wave containing information about a sound source. 
   
   
       15 . The system of  claim 14 , wherein the information of a sound wave includes at least one of a group of distance between a sound source and the microphone, latency of the sound wave, delay of the sound wave and phase shift of the sound wave. 
   
   
       16 . The system of  claim 10 , wherein the immersive sound processing module is further configured to reconcile the three-dimensional video model with the location of the sound source. 
   
   
       17 . The system of  claim 10 , wherein the immersive sound processing module is further configured to:
 add sounds from sound sources not contained in the three-dimensional video model; and   determine location of the added sound source;   
   
   
       18 . The system of  claim 17 , the immersive sound processing module is further configured to add the location of the added sound source to the first sound texture map to generate a composite sound texture map.