Method and apparatus for interactive real time music composition

Abstract

An interactive dynamic musical composition real time music presentation video game system uses individually composed musical compositions stored as building blocks. The building blocks are structured as nodes of a sequential state machine. Transitions between states are defined based on exit point of current state and entrance point into the new state. Game-related parameters can trigger transition from one compositional building block to another. For example, an interactivity variable can keep track of the current state of the video game or some aspect of it. In one example, an adrenaline counter gauging excitement based on the number of game objectives that have been accomplished can be used to control transitions between more relaxed musical states to more exciting and energetic musical states. Transitions can be handled by cross-fading between one music compositional component to another, or by providing transitional compositions. The system can be used to dynamically generate a musical composition in real time. Advantages include allowing a musical composer to compose a number of discrete musical compositions corresponding to different video game or other multimedia presentation states, and providing smooth transition between the different compositions responsive to interactive user input and/or other parameters.

We claim:  
     
       1. A computer-assisted sound generation method that uses a computer system to generate sounds with transitional variations the computer system dynamically introduces based on user interaction with the computer system, said method comprising: 
       defining plural predefined states of an associated state machine providing variable sequences of said states and at least some predefined conditions for transitioning between said states, at least some of said states of the state machine having an associated pre-defined music composition component and at least one predetermined exit point associated therewith;  
       defining an interactivity parameter responsive at least in part to user interaction with the computer system;  
       transitioning between said pre-defined states at said predetermined exit points based at least in part on the interactivity parameter; and  
       producing sound in response to a current said states and said transitions between said states such that said interactivity parameter at least in part dynamically selects, based on said predefined conditions, transitions between said musical composition components and associated produced sounds.  
     
     
       2. The method of  claim 1  wherein said interactivity parameter is responsive to a user input device. 
     
     
       3. The method of  claim 1  wherein each of said pre-defined music composition components comprises a MIDI file with loop back. 
     
     
       4. The method of  claim 1  wherein said transitioning is performed in response to state transition control data, said state transition control data predefining said conditions for transitioning between said states. 
     
     
       5. The method of  claim 4  wherein said state transition control data comprises at least one exit point and at least one entrance point per state. 
     
     
       6. The method of  claim 1  wherein said producing step is performed using, at least in part, a 3D graphics and audio processor. 
     
     
       7. The method of  claim 1  further comprising generating computer graphics associated with said states based at least in part on said interactivity parameter. 
     
     
       8. The method of  claim 1  wherein at least some of said music composition components comprise humanly-authored precomposed and performed musical components. 
     
     
       9. A computer system for dynamically generating sounds comprising: 
       a storage device that stores a plurality of musical compositions precomposed by a human being;  
       said storage device storing additional data assigning each of said plurality of musical compositions to a state of a state machine providing sequences of states and at least some predefined conditions for transitioning between said states and defining connections between said states;  
       at least one user-manipulable input device; and  
       a music engine responsive to said user-manipulable input device that transitions between different states of said state machine in response to user input, thereby dynamically generating a musical or other audio presentation based on user input by dynamically selecting between different precomposed musical compositions such that said user input at least in part dynamically selects transitions between said musical compositions.  
     
     
       10. The system of  claim 8  wherein at least one of said states is selected also based on a variable other than user interactivity. 
     
     
       11. The system of  claim 8  wherein each of said plurality of musical compositions is stored in a looping audio file. 
     
     
       12. The system of  claim 8  wherein at least some of said plurality of musical compositions and associated states are selected based at least in part on virtual weather conditions. 
     
     
       13. The method of  claim 8  wherein at least some of said states are selected based at least in part on an adrenaline factor indicating overall excitement level. 
     
     
       14. The system of  claim 8  wherein at least some of said states are selected based at least in part on success in accomplishing game play objectives. 
     
     
       15. The system of  claim 8  wherein at least some of said states are selected based at least in part on failure to accomplish game play objectives. 
     
     
       16. A method of dynamically producing sound effects to accompany video game play, said video game having an environment parameter, said method comprising: 
       defining at least one cluster of musical states and associated state transition connections therebetween, said cluster defining sequences of sound states and at least some predefined conditions for transitioning between said sound states based at least in part on interactive user input, at least some of said states having pre-composed sounds associated therewith;  
       accepting user input;  
       transitioning between said states within said cluster based at least in part on said accepted user input; and  
       transitioning between said states within said cluster and additional states outside of said cluster based at least in part on a video game environment parameter.  
     
     
       17. The method of  claim 16  wherein said video game environment parameter comprises a virtual weather indicator. 
     
     
       18. A method of generating music via computer of the type that accepts user input, said method comprising; 
       storing first and second sound files each encoding a respective precomposed musical piece, said sound files defining a state machine providing a sequence of states and at least some predefined conditions for transitioning between said states;  
       dynamically transitioning, in response to user input and under predefined transitioning conditions, between said first sound file and said second sound file by using a predetermined exit point of said first sound file and a predetermined entrance point of said second sound file; and  
       performing an additional transition between said first sound file and said second sound file via a third, bridging sound file providing a smooth transition between said first sound file and said second sound file.  
     
     
       19. The method of  claim 18  wherein at least one of said predetermined exit and entrance points is other than the beginning of the associated sound file, said predefined music composition components each comprising a portion of a musical composition precomposed by a human composer. 
     
     
       20. A method of generating interactive program material for a multimedia presentation comprising: 
       defining at least one cluster of states and associated state transition connections therebetween, said cluster defining sequences of states and predefined conditions for transitioning between said states based at least in part on interactive user input, said states each having programmable presentation material associated therewith;  
       accepting user input;  
       transitioning between said states within said cluster based at least in part on said accepted user input; and  
       transitioning between said states within said cluster and additional states outside of said cluster based at least in part on a variable multimedia presentation environment parameter other than said accepted user input to present a dynamic programmable multimedia presentation to the user that dynamically responds to said accepted user input.