Monitor-mine-manage cycle

Abstract

A monitor-mine-manage cycle is described, for example, for managing a data center, a manufacturing process, an engineering process or other processes. In various example, the following steps are performed as a continuous automated loop: receiving raw events from an observed system; monitoring the raw events and transforming them into complex events; mining the complex events and reasoning on results; making a set of proposed actions based on the mining; and managing the observed system by applying one or more of the proposed actions to the system. In various examples, the continuous automated loop proceeds while raw events are continuously received from the observed system and monitored. In some examples an application programming interface is described comprising programming statements which allow a user to implement a monitor-mine-manage loop.

The invention claimed is: 
     
       1. A computer-implemented method, comprising the following performed as a continuous automated loop:
 monitoring a plurality of events indicative of user request queue size fluctuations at an observed system; 
 extracting one or more features from the plurality of events; 
 aggregating the one or more features into one or more other events indicative of an average user request queue size fluctuation over time; 
 detecting a pattern in the average user request queue size fluctuation by mining the one or more other events; 
 determining alignment of the user request queue size fluctuations with a service level agreement, including:
 determining whether the detected pattern is indicative of service level agreement fulfillment; and 
 predicting, with a Bayesian model, subsequent queue size fluctuations; and 
 
 better aligning a configuration of a set of servers included in the observed system with the service level agreement, including one or more of: deploying a server to the observed system or removing a server from the observed system, while additional events are continuously monitored at the observed system. 
 
     
     
       2. The method of  claim 1 , wherein monitoring the plurality of events comprises monitoring the plurality of events in real-time or near real-time. 
     
     
       3. The method of  claim 1 , wherein monitoring the plurality of events at an observed system comprises monitoring events from the set of servers after a prior change to the configuration of servers in the set of servers. 
     
     
       4. The method of  claim 1 , wherein detecting a pattern in the average queue size fluctuation over time comprises performing at least one of statistical analysis, pattern recognition and machine learning on the one or more other events. 
     
     
       5. The method of  claim 1 , wherein determining alignment of the user request queue size fluctuations with a service level agreement comprises determining if a specified number of client requests per second are being fulfilled. 
     
     
       6. The method of  claim 1 , wherein determining alignment of the user request queue size fluctuations with a service level agreement comprises determining if one or more measurable details of the service level agreement are being satisfied, the one or more measurable details selected from among: mean time between failures, mean time to repair, mean time to recovery, throughput, and jitter. 
     
     
       7. The method of  claim 1 , further comprising performing one or more of: repairs, updates, backups and configurations on the observed system. 
     
     
       8. The method of  claim 1 , wherein better aligning a configuration of a set of servers with the service level agreement comprises better aligning the configuration of the set of servers in accordance with static constraints, including costs for the proposed actions. 
     
     
       9. The method of  claim 8 , wherein better aligning the configuration of the set of servers comprises better aligning the configuration of the set of servers with at least one of: a maximum number of servers deployed and a maximum number of electricity consumed per day. 
     
     
       10. A system, the system comprising:
 a processor; 
 system memory coupled to the processors and storing instructions configured to cause the processor to:
 monitor a plurality of events indicative of user request queue size fluctuations at an observed system; 
 extract one or more features from the plurality of events; 
 aggregate the one or more features into one or more other events indicative of an average user request queue size fluctuation over time; 
 detect a pattern in the average user request queue size fluctuation by mining the one or more other events; 
 determine alignment of the user request queue size fluctuations with a service level agreement, including:
 determine whether the detected pattern is indicative of service level agreement fulfillment; and 
 predict, with a Bayesian model, subsequent queue size fluctuations; and 
 
 better align a configuration of a set of servers included in the observed system with the service level agreement including one or more of: deploying a server to observed system or removing a server from the observed system, while additional events are continuously monitored at the observed system. 
 
 
     
     
       11. The system of  claim 10 , wherein the system is at least partially implemented at one or more of: a field-programmable gate array, a program-specific integrated circuit, a program-specific standard product, a system-on-a-chip, or a complex programmable logic device. 
     
     
       12. The system of  claim 10 , wherein instructions configured to cause the processor to better align a configuration of a set of servers with the service level agreement comprise instructions configured to cause the processor to optimize cost by removing a server from the set of servers. 
     
     
       13. The system of  claim 10 , wherein instructions configured to cause the processor to better align a configuration of a set of servers with the service level agreement comprise instructions configured to cause the processor to ensure that the service level agreement is satisfied by deploying a server to the set of servers. 
     
     
       14. The system of  claim 10 , wherein instructions configured to cause the processor to determine alignment of the user request queue size fluctuations with a service level agreement comprise instructions configured to cause the processor to determine if one or more measurable details of the service level agreement are being satisfied, the one or more measurable details selected from among: mean time between failures, mean time to repair, mean time to recovery, throughput, and jitter.