Image stream pipeline controller for deploying image primitives to a computation fabric

Abstract

According to some embodiments, an image pipeline controller may determine an image stream having a plurality of image primitives to be executed. Each image primitive may be, for example, associated with an image algorithm and a set of primitive attributes. The image pipeline controller may then automatically deploy the set of image primitives to an image computation fabric based at least in part on primitive attributes.

1 - 29 . (canceled) 
     
     
         30 . A method, comprising
 generating, with a run-time framework, a plurality of image primitives grouped into image segments, wherein at least one of the image primitives to create output image data;   wherein at least one image primitive is associated with an image primitive library, the image primitives to be at least one of: (i) a histogram primitive, (ii) a scaling primitive; or (iii) a machine vision primitive; and   deploying, by the run-time framework, the plurality of image primitives to hardware for execution.   
     
     
         31 . The method of  claim 30 , wherein image segments are grouped for at least one of in-order execution or out-of-order execution. 
     
     
         32 . The method of  claim 30 , wherein image segments are grouped in the run-time framework for in-order execution and out-of-order execution, wherein a first run-time framework is executed in order and a second run-time framework is executed out of order. 
     
     
         33 . The method of  claim 30 , wherein the output image data is output to a display. 
     
     
         34 . The method of  claim 30 , wherein the image primitives generate output image data from input image input received from an image sensor. 
     
     
         35 . The method of  claim 30 , wherein the generation of the run-time framework is powered by a battery power source. 
     
     
         36 . The method of  claim 30 , wherein the run-time framework comprises at least one of: (i) a hardware run-time framework, (ii) a software run-time framework, or (iii) a combination of hardware and software run-time framework components. 
     
     
         37 . The method of  claim 30 , wherein image primitives associated with a plurality of image segments are deployed. 
     
     
         38 . The method of  claim 30 , wherein at least one of the segments is executed by an operating system, and information about the segments is associated with an application programming interface. 
     
     
         39 . The method of  claim 30 , further comprising the hardware for execution comprises at least one of a system on a chip, a computation fabric, a processing unit, or a fixed function hardware image processing unit 
     
     
         40 . The method of  claim 30 , further comprising executing a sequencing algorithm to order the image primitives within an image segment for an in-order image primitive execution in the run-time framework. 
     
     
         41 . The method of  claim 30 , wherein the image primitives comprise an original order and said executing is performed for at least some of the image primitives in an order different than the original order for an out-of-order image primitive execution in the run-time framework. 
     
     
         42 . The method of  claim 30 , wherein the output image data is produced by the image primitives before any reader of that data accesses it. 
     
     
         43 . The method of  claim 30 , wherein at least one image segment comprises at least one of: (i) pixel correction, (ii) artifact removal, (iii) histogram information, (iv) a scaling function, (v) face recognition, (vi) visual object recognition, (vii) visual scene analysis, (viii) machine vision, (ix) gesture recognition, or (x) depth map calculation. 
     
     
         44 . A non-transitory computer-readable storage on at least one medium having stored thereon instructions that when executed:
 generate, with a run-time framework, a plurality of image primitives grouped into image segments, wherein at least one of the image primitives to create output image data;   wherein at least one image primitive is associated with an image primitive library, the image primitives to be at least one of: (i) a histogram primitive, (ii) a scaling primitive; or (iii) a machine vision primitive; and   deploy, by the run-time framework, the plurality of image primitives to hardware for execution.   
     
     
         45 . The non-transitory computer-readable storage of  claim 44 , wherein image segments are grouped for at least one of in-order execution and out-of-order execution. 
     
     
         46 . The non-transitory computer-readable storage of  claim 44 , wherein image segments are grouped in the run-time framework for in-order execution and out-of-order execution, wherein a first run-time framework is executed in order and a second run-time framework is executed out of order. 
     
     
         47 . The non-transitory computer-readable storage of  claim 44 , wherein the output image data is output to a display. 
     
     
         48 . The non-transitory computer-readable storage of  claim 44 , wherein the image primitives generate output image data from input received from a camera. 
     
     
         49 . The non-transitory computer-readable storage of  claim 44 , wherein generation of the run-time framework is powered by a battery power source. 
     
     
         50 . The non-transitory computer-readable storage of  claim 44 , wherein the run-time framework comprises at least one of: (i) a hardware run-time framework, (ii) a software run-time framework, or (iii) a combination of hardware and software run-time framework components. 
     
     
         51 . The non-transitory computer-readable storage of  claim 44 , wherein image primitives are associated with a plurality of image segments are deployed. 
     
     
         52 . The non-transitory computer-readable storage of  claim 44 , wherein at least one of the segments is executed by an operating system, and information about the segments is associated with an application programming interface. 
     
     
         53 . The non-transitory computer-readable storage of  claim 44 , wherein the hardware for execution comprises at least one of a system on a chip, a computation fabric, a processing unit, or a fixed function hardware image processing unit. 
     
     
         54 . The non-transitory computer-readable storage of  claim 44 , further comprising executing a sequencing algorithm to order the image primitives within an image segment for an in-order image primitive execution in the run-time framework. 
     
     
         55 . The non-transitory computer-readable storage of  claim 44 , wherein the image primitives comprise an original order and said executing is performed for at least some of the image primitives in an order different than the original order for an out-of-order image primitive execution in the run-time framework. 
     
     
         56 . The non-transitory computer-readable storage of  claim 44 , wherein the output image data is produced by the image primitives before any reader of that data accesses it. 
     
     
         57 . The non-transitory computer-readable storage of  claim 44 , wherein at least one image segment comprises at least one of: (i) pixel correction, (ii) artifact removal, (iii) histogram information, (iv) a scaling function, (v) face recognition, (vi) visual object recognition, (vii) visual scene analysis, (viii) machine vision, (ix) gesture recognition, or (x) depth map calculation. 
     
     
         58 . A system, comprising:
 a processor;   an image sensor to create input image data; and   a memory to store instructions that executed by the processor:
 generate, with a run-time framework, a plurality of image primitives grouped into image segments, wherein at least one of the image primitives to create output image data in response to the input image data, wherein at least one image primitive is associated with an image primitive library, the image primitives to be at least one of: (i) a histogram primitive, (ii) a scaling primitive; or (iii) a machine vision primitive; and 
 deploy, by the run-time framework, the plurality of image primitives to hardware for execution. 
   
     
     
         59 . The system of  claim 58 , wherein image segments are grouped for at least one of in-order execution and out-of-order execution. 
     
     
         60 . The system of  claim 58 , wherein image segments are grouped in the run-time framework for in-order execution and out-of-order execution, wherein a first run-time framework is executed in order and a second run-time framework is executed out of order. 
     
     
         61 . The system of  claim 58 , wherein the image primitives generate output image data from input received from a camera. 
     
     
         62 . The system of  claim 58 , wherein generation of the run-time framework is powered by a battery power source. 
     
     
         63 . The system of  claim 58 , wherein the run-time framework comprises at least one of: (i) a hardware run-time framework, (ii) a software run-time framework, or (iii) a combination of hardware and software run-time framework components. 
     
     
         64 . The system of  claim 58 , wherein image primitives associated with a plurality of image segments are deployed. 
     
     
         65 . The system of  claim 58 , wherein at least one of the segments is executed by an operating system, and information about the segments is associated with an application programming interface. 
     
     
         66 . The system of  claim 58 , wherein the hardware for execution comprises at least one of a system on a chip, a computation fabric, a processing unit, or a fixed function hardware image processing unit. 
     
     
         67 . The system of  claim 58 , further comprising executing a sequencing algorithm to order the image primitives within an image segment for an in-order image primitive execution in the run-time framework. 
     
     
         68 . The system of  claim 58 , wherein the image primitives comprise an original order and said executing is performed for at least some of the image primitives in an order different than the original order for an out-of-order image primitive execution in the run-time framework. 
     
     
         69 . The system of  claim 58 , wherein the output image data is produced by the image primitives before any reader of that data accesses it. 
     
     
         70 . The system of  claim 58 , wherein at least one image segment comprises at least one of: (i) pixel correction, (ii) artifact removal, (iii) histogram information, (iv) a scaling function, (v) face recognition, (vi) visual object recognition, (vii) visual scene analysis, (viii) machine vision, (ix) gesture recognition, or (x) depth map calculation. 
     
     
         71 . A system, comprising:
 a means to process information;   a means for creating input image data; and   a means to store instructions that executed by the processor:
 generate a plurality of image primitives grouped into image segments to create output image data in response to the input image data, wherein at least one image primitive is associated with an image primitive library, the image primitives to be at least one of: (i) a histogram primitive, (ii) a scaling primitive; or (iii) a machine vision primitive; and 
 deploy the plurality of image primitives to means for execution. 
   
     
     
         72 . The system of  claim 71 , wherein image segments are grouped for at least one of in-order execution and out-of-order execution. 
     
     
         73 . The system of  claim 71 , wherein image segments are grouped for in-order execution and out-of-order execution, wherein a first means of execution executes image segments in order and a second means of execution executes image segments out-of-order. 
     
     
         74 . The system of  claim 71 , wherein the image primitives generate output image data from input received from a means for capturing input. 
     
     
         75 . The system of  claim 71 , wherein generation of means of execution for image segments is powered by means of suppling power. 
     
     
         76 . The system of  claim 71 , wherein at least one of the segments is executed by an operating system, and information about the segments is associated with an application programming interface. 
     
     
         77 . The system of  claim 71 , wherein the hardware for execution comprises at least one of a system on a chip, a computation fabric, a processing unit, or a fixed function hardware image processing unit. 
     
     
         78 . The system of  claim 71 , further comprising executing a sequencing algorithm to order the image primitives within an image segment for an in-order image primitive execution in means of run-time execution. 
     
     
         79 . The system of  claim 71 , wherein the image primitives comprise an original order and said executing is performed for at least some of the image primitives in an order different than the original order for an out-of-order image primitive execution in means of run-time execution. 
     
     
         80 . The system of  claim 71 , wherein the output image data is produced by the image primitives before any reader of that data accesses it. 
     
     
         81 . The system of  claim 71 , wherein at least one image segment comprises at least one of: (i) pixel correction, (ii) artifact removal, (iii) histogram information, (iv) a scaling function, (v) face recognition, (vi) visual object recognition, (vii) visual scene analysis, (viii) machine vision, (ix) gesture recognition, or (x) depth map calculation.