System for Optimizing Graphics Operations

Abstract

Disclosed is a system for producing images including techniques for reducing the memory and processing power required for such operations. The system provides techniques for programmatically representing a graphics problem. The system further provides techniques for reducing and optimizing graphics problems for rendering with consideration of the system resources, such as the availability of a compatible GPU.

1 . A non-transitory computer readable medium comprising computer instructions stored thereon to create an image utilizing at least one or more central processing units (CPUs) and one or more graphics processing units (GPUs), the computer instructions comprising instructions to cause two or more processors to:
 represent an image in memory using an image graph comprising two or more GPU programs, wherein the two or more GPU programs are represented as nodes in the image graph and inputs to the two or more GPU programs and outputs from the two or more GPU programs are represented as links in the image graph;   generate an optimized image graph by running software one of the one or more CPUs wherein optimizing comprises analyzing with a CPU whether two GPU programs may be combined, and performing a combination if said analysis is positive and wherein analyzing comprises analysis of inputs to the two GPU programs and outputs from the two GPU programs;   compile said optimized image graph by running software on one of the one or more CPUs to create a compiled image graph; and   render a graphical result from said image graph on a display by running said compiled image graph on the one or more GPUs.   
     
     
         2 . The non-transitory computer readable medium of  claim 1 , wherein the instructions to cause the two or more processors to generate an optimized image graph include instructions to cause one of the two or more processors to perform a cache look-up to see if at least a portion of said optimized image graph is already in cache. 
     
     
         3 . The non-transitory computer readable medium of  claim 1 , wherein the instructions to cause the two or more processors to generate an optimized image graph include instructions to cause one of the two or more processors to calculate an intersection, said intersection representing an area where said rendered image is both defined by said image graph and part of a region requested by a process running on one of the one or more CPUs that have requested creation of said image. 
     
     
         4 . The non-transitory computer readable medium of  claim 2 , wherein the instructions to cause the two or more processors to generate an optimized image graph include instructions to cause one of the two or more processors to calculate an intersection, said intersection representing an area where said rendered image is both defined by said image graph and part of a region requested by a process running on one of the one or more CPUs that have requested creation of said image. 
     
     
         5 . The non-transitory computer readable medium of claim method of  claim 3  further comprising instructions to cause one of the two or more processors to use said calculated intersection to limit the number of pixels that require calculation during said rendering by said one or more GPUs. 
     
     
         6 . The non-transitory computer readable medium of claim method of  claim 4  further comprising instructions to cause one of the two or more processors to use said calculated intersection to limit the number of pixels that require calculation during said rendering by said one or more GPUs. 
     
     
         7 . The non-transitory computer readable medium of claim method of  claim 3  further comprising instructions to cause one of the two or more processors to use said calculated intersection to limit the amount of memory necessary for storing said rendered image. 
     
     
         8 . The non-transitory computer readable medium of claim method of  claim 4  further comprising instructions to cause one of the two or more processors to use said calculated intersection to limit the amount of memory necessary for storing said rendered image. 
     
     
         9 . The non-transitory computer readable medium of  claim 1 , wherein the instructions to cause the one or more processors to generate an optimized image graph further comprise instructions to cause one of the two or more processors to use one of the one or more CPUs to determine if one of said one or more GPUs is capable of running a program that has been created by combining two other programs. 
     
     
         10 . A non-transitory computer readable medium comprising computer instructions stored thereon to cause two or more processors to perform a method for creating a rendered polygon using an image graph based representation of graphics processing unit (GPU) programs, the computer instructions comprising instructions to cause the two or more processors to:
 receive a request to render a polygon at one of one or more central processing units (CPUs);   create, on the one or more CPUs, an image graph based representation of GPU programs to create said rendered polygon, the image graph based representation comprising a root GPU program and its relationship with other GPU programs, wherein said GPU programs are represented as nodes with their inputs and outputs represented as links;   analyze whether two GPU programs may be combined, and performing a combination if said analysis is positive wherein analyzing comprises analysis of inputs to the two GPU programs and outputs from the two GPU programs; and   create a buffer and cause a GPU to render an image to that buffer by running a GPU program.   
     
     
         11 . The non-transitory computer readable medium of  claim 10 , wherein said image graph based representation comprises a low-level image graph. 
     
     
         12 . The non-transitory computer readable medium of  claim 10 , wherein said image graph based representation comprises a high-level image graph. 
     
     
         13 . The non-transitory computer readable medium of  claim 10  further comprising instructions to cause the two or more processors to allow an application program under CPU control to make said request to render said polygon. 
     
     
         14 . The non-transitory computer readable medium of  claim 10 , wherein the instructions to cause the two or more processors to analyze whether two GPU programs may be combined, and performing a combination if said analysis is positive, further comprise instructions implemented to be recursively called until no more combinations are possible. 
     
     
         15 . The non-transitory computer readable medium of  claim 10  further comprising instructions to cause one of the two or more processors to perform domain of definition (DOD) and region of interest (ROI) optimization wherein the DOD/ROI optimization comprises intersecting the ROI with the output DOD for a GPU program. 
     
     
         16 . The non-transitory computer readable medium of  claim 10  further comprising instructions to cause one of the two or more processors to use a cache in order to find the result of running a particular GPU program, without running the particular GPU program. 
     
     
         17 . A method for creating a rendered polygon using an image graph based representation of graphics processing unit (GPU) programs, the method comprising the steps of:
 under control of a central processing unit (CPU), receiving a request to render a polygon;   under control of said CPU, creating an image graph based representation of GPU programs to create said rendered polygon, the image graph based representation comprising a root GPU program and its relationship with other GPU programs, wherein said GPU programs are represented as nodes with their inputs and outputs represented as links;   analyzing whether two GPU programs may be combined, and performing a combination if said analysis is positive wherein analyzing comprises analysis of inputs to the two GPU programs and outputs from the two GPU programs; and   creating a buffer and causing a GPU to render an image to that buffer by running at least one combined GPU program.   
     
     
         18 . A method of creating an image utilizing at least a central processing unit (CPU) and a graphics processing unit (GPU), the method comprising:
 representing, in memory, an image by an image graph wherein the image graph comprises one or more GPU programs represented as nodes in the image graph, inputs to the one or more GPU programs and outputs from the one or more GPU programs wherein the inputs and outputs are represented as links in the image graph;   optimizing said image graph by running software on a CPU wherein optimizing comprises creating an optimized image graph, wherein optimizing said image graph comprises analyzing with said CPU whether two GPU programs may be combined, and performing a combination if said analysis is positive and wherein analyzing comprises analysis of inputs to the two GPU programs and outputs from the two GPU programs;   compiling said optimized image graph by running software on said CPU to create a compiled image graph; and   rendering said image graph by running said compiled image graph on a GPU, yielding a rendered image.   
     
     
         19 . A non-transitory computer readable medium comprising computer instructions stored thereon to create an image utilizing two or more processors, wherein at least one of the two or more processors include a central processing unit (CPU) and another of the two or more processors include a graphics processing unit (GPU), the computer instructions comprising instructions to cause the two or more processors to:
 represent an image in memory using an image graph comprising two or more GPU programs, wherein the two or more GPU programs are represented as nodes in the image graph and inputs to the two or more GPU programs and outputs from the two or more GPU programs are represented as links in the image graph;   generate an optimized image graph by running software on a CPU to combine two of the two or more GPU programs when analysis by the CPU of the inputs to, and the outputs from, the two GPU programs so indicate;   generate a compiled image graph by running software on the CPU; and   render the image on a display by running the compiled image graph on a GPU.   
     
     
         20 . The non-transitory computer readable medium of  claim 19 , wherein the instructions to cause the two or more processors to generate an optimized image graph include instructions to cause one of the two or more processors to perform a cache look-up to see if at least a portion of said optimized image graph is already in cache. 
     
     
         21 . The non-transitory computer readable medium of  claim 19 , wherein the instructions to cause the two or more processors to generate an optimized image graph include instructions to cause one of the two or more processors to calculate an intersection, said intersection representing an area where said rendered image is both defined by said image graph and part of a region requested by a process running on one of one or more CPUs that have requested creation of said image. 
     
     
         22 . The non-transitory computer readable medium of claim method of  claim 21  further comprising instructions to cause one of the two or more processors to use said calculated intersection to limit the number of pixels that require calculation during said rendering by said one or more GPUs. 
     
     
         23 . The non-transitory computer readable medium of  claim 19 , wherein the instructions to cause the one or more processors to generate an optimized image graph further comprise instructions to cause one of the two or more processors to use one of the one or more CPUs to determine if one of said one or more GPUs is capable of running a program that has been created by combining two other programs.