Object-oriented multi-media architecture

Abstract

An object-oriented, multi-media architecture provides for real-time processing of an incoming stream of pseudo-language byte codes compiled from an object-oriented source program. The architecture includes a plurality of processors arranged for parallel processing. At least some of the processors are especially adapted or optimized for execution of multi-media methods such as video decompression, inverse discrete cosine transformation, motion estimation and the like. The architecture further includes a virtual machine computer program that reconstructs objects and threads from the byte code stream, and routes each of them to the appropriate hardware resource for parallel processing. This architecture extends the object-oriented paradigm through the operating system and execution hardware of a client machine to provide the advantages of dedicated/parallel processors while preserving portability of the pseudo-language environment.

I claim: 
     
       1. A system for real-time processing of an incoming stream of pseudo-language byte codes compiled from an object-oriented source program, the system comprising: a memory;   input means for receiving the incoming stream of byte codes;   a plurality of hardware processors arranged in parallel; and   a virtual machine computer program stored in the memory and coupled to the input means and including: means for reconstructing a software thread from the stream of byte codes;   means for translating the reconstructed software thread into a corresponding executable process;   means for selecting an appropriate one of the hardware processors for executing the corresponding executable process; and   means for routing the corresponding executable process to the selected hardware processor for execution.     
     
     
       2. A system according to claim 1 wherein the virtual machine program includes means for maintaining a list of hardware processors currently available in the system and means for selecting an appropriate one of the hardware processors for executing the executable process includes means for examining the list, whereby the system automatically adapts so as to take advantage of currently available hardware processors. 
     
     
       3. A system according to claim 1 wherein the virtual machine program includes a library of predetermined class protocols stored in the memory and the means for selecting the hardware processor includes means for correlating the list of hardware processors currently available in the system to the library of object class protocols. 
     
     
       4. A system according to claim 3 wherein the virtual machine program includes dynamic class loader means for determining whether an object referenced in the incoming byte code stream is an instance of a class protocol defined in the class library and for downloading any missing class protocol or library. 
     
     
       5. A system according to claim 1 wherein the virtual machine program includes means for reconstructing a plurality of distinct process threads from the byte codes, means for selecting a respective hardware processor for execution of each of the process threads, and means for routing each of the process threads to the corresponding selected hardware processor for parallel execution as lightweight processes. 
     
     
       6. A system according to claim 5 wherein the processors include at least one native byte code processor dedicated to executing native byte codes, and the virtual machine includes means for routing a flat process thread to the native byte code processor for execution without translating op codes. 
     
     
       7. A system according to claim 5 and further comprising a micro-kernel operating system for interfacing between the hardware processors and the virtual machine program. 
     
     
       8. A system according to claim 7 wherein the virtual machine program is written in an object-oriented programming language. 
     
     
       9. A system according to claim 8 wherein the object-oriented programming language; includes classes of the Java programming language and the virtual machine program includes a library of Java class protocols stored in the memory; and, the means for selecting the hardware processor includes means for correlating the list of hardware processors currently available in the system to the library of Java class protocols. 
     
     
       10. A byte-code interpreted, hardware neutral, computer-implemented virtual machine method of converting an input stream of byte codes compiled from an object-oriented, multi-media source program into a plurality of process threads for parallel execution in real-time, the method comprising the steps of: maintaining a list of available hardware resources;   reconstructing a plurality of program objects and threads from the input stream of byte codes;   correlating the reconstructed program objects and threads to a list of the available hardware resources; and   routing each of the reconstructed program objects and threads to a respective selected one of the hardware resources for substantially parallel execution, thereby providing improved execution performance while maintaining portability of the byte codes.   
     
     
       11. A method according to claim 10 further comprising: identifying processes declared synchronized in the source program; and   executing the said synchronized processes under control of monitors so as to ensure that variables remain in a consistent state; and   scheduling all other processes for execution in parallel to the extent that the available hardware resources permit.   
     
     
       12. A byte code interpreted, hardware neutral, computer-implemented virtual machine method of converting an input stream of byte codes compiled from an object-oriented, multi-media source program into a plurality of process threads for parallel execution in real-time, the method comprising the steps of: maintaining a list of available hardware resources;   reconstructing a plurality of program objects and threads from the input stream of byte codes;   correlating the reconstructed program objects and threads to a list of the available hardware resources;   routing each of the reconstructed program objects and threads to a respective selected one of the hardware resources for substantially parallel execution, thereby providing improved execution performance while maintaining portability of the byte codes; and   supporting predetermined synchronization methods defined in the multi-media application, the synchronization methods allowing multiple threads within the multi-media application access to and exclusive control of shared resources.   
     
     
       13. A system for real-time processing of an incoming stream of object-oriented, multi-media data comprising: input means for receiving the incoming stream of data;   a plurality of hardware processors arranged in parallel;   a virtual machine computer program coupled to the input means and including means for reconstructing a software thread from the input data, means for translating the reconstructed software thread into a corresponding executable process, means for selecting an appropriate one of the hardware processors for executing the corresponding executable process, and means for routing the corresponding executable process to the selected hardware processor for execution; and   a micro-kernel operating system for managing the plurality of hardware processors, the micro-kernel operating system including means for surveying the hardware processors so as to provide a list of available processors to the virtual machine program for use in connection with said selecting an appropriate one of the hardware processors for executing the corresponding executable process.   
     
     
       14. A system according to claim 13 wherein the hardware processors include at least one specialized processor optimized for executing a predetermined method of a multi-media object and at least one RISC processor for executing a flat code segment in a native pseudo-language. 
     
     
       15. A system for real-time processing of an incoming stream of pseudo-language byte codes compiled from an object-oriented source program, the system comprising: a memory;   input means for receiving the incoming stream of byte codes;   a plurality of hardware processors arranged in parallel, wherein the processors include at least one multi-media processor dedicated to execution of a selected multi-media method; and   a virtual machine computer program stored in the memory and coupled to the input means and including: means for reconstructing a plurality of distinct process threads from the byte codes;   means for translating the reconstructed software threads into corresponding executable processes;   means for selecting a respective hardware processor for execution of each of the process threads;   means for routing each of the process threads to the corresponding selected hardware processor for parallel execution as lightweight processes; and   means for routing a multi-media process thread to the multi-media processor.     
     
     
       16. A system for real-time processing of an incoming stream of pseudo-language byte codes compiled from an object-oriented source program, the system comprising: a memory;   input means for receiving the incoming stream of byte codes;   a plurality of hardware processors arranged in parallel, wherein the processors include at least one audio processor dedicated to execution of audio data; and   a virtual machine computer program stored in the memory and coupled to the input means and including: means for reconstructing a plurality of distinct process threads from the byte codes;   means for translating the reconstructed software threads into corresponding executable processes;   means for selecting a respective hardware processor for execution of each of the process threads;   means for routing each of the process threads to the corresponding selected hardware processor for parallel execution as lightweight processes; and   means for routing an audio process thread to the audio processor.     
     
     
       17. A system for real-time processing of an incoming stream of pseudo-language byte codes compiled from an object-oriented source program, the system comprising: a memory;   input means for receiving the incoming stream of byte codes;   a plurality of hardware processors arranged in parallel, wherein the processors include at least one graphics decompression processor dedicated to decompression of graphics data; and   a virtual machine computer program stored in the memory and coupled to the input means and including: means for reconstructing a plurality of distinct process threads from the byte codes;   means for translating the reconstructed software threads into corresponding executable processes;   means for selecting a respective hardware processor for execution of each of the process threads;   means for routing each of the process threads to the corresponding selected hardware processor for parallel execution as lightweight processes; and   means for routing a graphics process thread to the graphics decompression processor.     
     
     
       18. A system for real-time processing of an incoming stream of pseudo-language byte codes compiled from an object-oriented source program, the system comprising: a memory;   input means for receiving the incoming stream of byte codes;   a plurality of hardware processors arranged in parallel, wherein the processors include at least one graphics rendering processor dedicated to rendering graphics data; and   a virtual machine computer program stored in the memory and coupled to the input means and including: means for reconstructing a plurality of distinct process threads from the byte codes;   means for translating the reconstructed software threads into corresponding executable processes;   means for selecting a respective hardware processor for execution of each of the process threads;   means for routing each of the process threads to the corresponding selected hardware processor for parallel execution as lightweight processes; and   means for routing a graphics process thread to the graphics rendering processor.     
     
     
       19. A byte code interpreted, hardware neutral, computer-implemented virtual machine method of converting an input stream of byte codes compiled from an object-oriented, multi-media source program into a plurality of process threads for parallel execution in real-time, the method comprising the steps of: maintaining a list of available hardware resources, wherein the list of available hardware resources includes, for each hardware resource, an indication of a specific multi-media function, if any, for which the hardware resource is specially adapted;   reconstructing a plurality of program objects and threads from the input stream of byte codes;   correlating the reconstructed program objects and threads to the list of the available hardware resources, including comparing the program objects and threads to the list so as to identify those hardware resources, if any, that are specially adapted to executing any of the program objects and threads; and   routing each of the reconstructed program objects and threads to a respective selected one of the hardware resources for substantially parallel execution, thereby providing improved execution performance while maintaining portability of the byte codes, including binding the program objects and threads to the corresponding identified hardware resources for execution.   
     
     
       20. A byte code interpreted, hardware neutral, computer-implemented virtual machine method of converting an input stream of byte codes compiled from an object-oriented, multi-media source program into a plurality of process threads for parallel execution in real-time, the method comprising the steps of: maintaining a list of available hardware resources, wherein the hardware resources include a plurality of processors, each processor optimized for its primary function;   reconstructing a plurality of program objects and threads from the input stream of byte codes;   correlating the reconstructed program objects and threads to the list of the available hardware resources;   routing each of the reconstructed program objects and threads to a respective selected one of the hardware resources for substantially parallel execution, thereby providing improved execution performance while maintaining portability of the byte codes; and   providing a coherent, global cache memory coupled to each of a plurality of the processors to improve performance.