System and method of stream processing workflow composition using automatic planning

Abstract

An automatic planning system is provided for stream processing workflow composition. End users provide requests to the automatic planning system. The requests are goal-based problems to be solved by the automatic planning system, which then generates plan graphs to form stream processing applications. A scheduler deploys and schedules the stream processing applications for execution within an operating environment. The operating environment then returns the results to the end users.

1 . A method, in a stream processing system, for composing stream processing workflows using automatic planning, wherein the stream processing system receives one or more primal streams and executes stream processing applications in a stream processing operating environment, the method comprising: 
 receiving a request for stream processing;    translating the request for stream processing into a formal expression of the request in a description language; and    generating a workflow based on the formal expression of the request and a domain definition in the description language, wherein the domain definition describes the stream processing operating environment, and wherein the workflow comprises nodes corresponding to stream processing application components with possible parameters values set and links corresponding to streams.    
   
   
       2 . The method of  claim 1 , further comprising: 
 adapting the workflow into a stream processing application that is executable in the stream processing operating environment.    
   
   
       3 . The method of  claim 2 , further comprising: 
 deploying the stream processing application to the stream processing operating environment.    
   
   
       4 . The method of  claim 1 , further comprising: 
 storing the domain definition in a planning cache.    
   
   
       5 . The method of  claim 1 , further comprising: 
 receiving a change notification that indicates one or more changes to the domain definition; and    adjusting the workflow based on the one or more changes to the domain definition to form an adjusted workflow.    
   
   
       6 . The method of  claim 5 , further comprising: 
 adapting the adjusted workflow into an adjusted stream processing application that is executable in the operating environment.    
   
   
       7 . The method of  claim 6 , further comprising: 
 deploying the adjusted stream processing application to the operating environment.    
   
   
       8 . The method of  claim 5 , further comprising: 
 determining a new domain definition in response to the change notification; and    storing the new domain definition in the planning cache.    
   
   
       9 . A stream processing system, for composing stream processing workflows using automatic planning, wherein the stream processing system receives one or more primal streams and executes stream processing applications, the stream processing system comprising: 
 a stream processing operating environment;    a controller configured to receive a request for stream processing;    a translation service configured to translate the request for stream processing into a formal expression of the request in a description language; and    a planning library configured to generate a workflow based on the formal expression of the request and a domain definition in the description language, wherein the domain definition describes the stream processing operating environment, and wherein the workflow comprises nodes corresponding to stream processing application components with possible parameters values set and links corresponding to streams.    
   
   
       10 . The stream processing system of  claim 9 , further comprising: 
 a plan adapter configured to adapt the workflow into a stream processing application that is executable in the stream processing operating environment.    
   
   
       11 . The stream processing system of  claim 10 , further comprising: 
 a scheduler configured to deploy the stream processing application to the stream processing operating environment.    
   
   
       12 . The stream processing system of  claim 9 , a planning cache configured to store the domain definition in a planning cache.  
   
   
       13 . The stream processing system of  claim 9 , wherein the controller is configured to receive a change notification that indicates one or more changes to the domain definition; and 
 wherein the planning library is configured to adjust the workflow based on the one or more changes to the domain definition to form an adjusted workflow.    
   
   
       14 . The stream processing system of  claim 13 , wherein the plan adapter is configured to adapt the adjusted workflow into an adjusted stream processing application that is executable in the operating environment.  
   
   
       15 . The stream processing system of  claim 14 , wherein the scheduler is configured to deploy the adjusted stream processing application to the operating environment.  
   
   
       16 . The stream processing system of  claim 13 , wherein the planning cache stores a new domain definition responsive to receipt of a change notification indicating a change in system state, user policy or profile, or component description.  
   
   
       17 . A computer program product for composing stream processing workflows using automatic planning, the computer program product comprising: 
 a computer usable medium having computer usable program code embodied therein;    computer usable program code configured to receive a request for stream processing in a stream processing system, wherein the stream processing system receives one or more primal streams and executes stream processing applications in a stream processing operating environment;    computer usable program code configured to translate the request for stream processing into a formal expression of the request in a description language; and    computer usable program code configured to generate a workflow based on the formal expression of the request and a domain definition in the description language, wherein the domain definition describes the stream processing operating environment, and wherein the workflow comprises nodes corresponding to stream processing application components with possible parameters values set and links corresponding to streams.    
   
   
       18 . The computer program product of  claim 17 , further comprising: 
 computer usable program code configured to adapt the workflow into a stream processing application that is executable in the stream processing operating environment.    
   
   
       19 . The computer program product of  claim 18 , further comprising: 
 computer usable program code configured to deploy the stream processing application to the stream processing operating environment.    
   
   
       20 . The computer program product of  claim 17 , further comprising: 
 computer usable program code configured to receive a change notification that indicates one or more changes to the domain definition; and    computer usable program code configured to adjust the workflow based on the one or more changes to the domain definition to form an adjusted workflow.