Controlling a Target System

Abstract

For controlling a target system, such as a gas or wind turbine or another technical system, a pool of control policies is used. The pool of control policies including a plurality of control policies and weights for weighting each control policy of the plurality of control policies are received. The plurality of control policies is weighted by the weights to provide a weighted aggregated control policy. The target system is controlled using the weighted aggregated control policy, and performance data relating to a performance of the controlled target system is received. The weights are adjusted based on the received performance data to improve the performance of the controlled target system. The plurality of control policies is reweighted by the adjusted weights to adjust the weighted aggregated control policy.

1 . A method for controlling a target system by a processor based on a pool of control policies, the method comprising:
 receiving the pool of control policies, the pool of control policies comprising a plurality of control policies;   receiving weights for weighting each control policy of the plurality of control policies;   weighting the plurality of control policies by the weights to provide a weighted aggregated control policy;   controlling the target system using the weighted aggregated control policy;   receiving performance data relating to a performance of the controlled target system;   adjusting the weights by the processor based on the received performance data to improve the performance of the controlled target system; and   reweighting the plurality of control policies by the adjusted weights to adjust the weighted aggregated control policy.   
     
     
         2 . The method of  claim 1 , wherein adjusting the weights comprises training a neural network run by the processor. 
     
     
         3 . The method of  claim 2 , further comprising:
 receiving operational data of at least one source system; and   calculating the plurality of control policies from different data sets of the operational data.   
     
     
         4 . The method of  claim 3 , wherein calculating the plurality of control policies comprises training the neural network or a further neural network. 
     
     
         5 . The method of  claim 3 , wherein calculating the plurality of control policies comprises using a reward function relating to a performance of the at least on source system, and
 wherein adjusting the weights comprises using the reward function for the adjusting of the weights.   
     
     
         6 . The method of  claim 1 , wherein the performance data comprises state data relating to a current state of the target system, and
 wherein the weighting of the plurality of control policies, the reweighting of the plurality of control policies, or the weighting of the plurality of control policies and the reweighting of the plurality of control policies depends on the state data.   
     
     
         7 . The method as claimed in  claim 1 , wherein receiving the performance data comprises receiving the performance data from the controlled target system, from a simulation model of the target system, from a policy evaluation, or from any combination thereof. 
     
     
         8 . The method of  claim 1 , wherein controlling the target system comprises determining an aggregated control action according to the weighted aggregated control policy by weighted majority voting, by forming a weighted mean, by forming a weighted median from action proposals according to the plurality of control policies, or by any combination thereof. 
     
     
         9 . The method of  claim 2 , wherein the training of the neural network is based on a reinforcement learning model. 
     
     
         10 . The method of  claim 2 , wherein the neural network operates as a recurrent neural network. 
     
     
         11 . The method of  claim 1 , wherein the plurality of control policies is selected from the pool of control policies in dependence of a performance evaluation of control policies. 
     
     
         12 . The method of  claim 1 , wherein control policies from the pool of control policies are included into or excluded from the plurality of control policies in dependence of the adjusted weights. 
     
     
         13 . The method of  claim 1 , wherein the controlling, the receiving of the performance data, the adjusting, and the reweighting are run in a closed learning loop with the target system. 
     
     
         14 . A controller for controlling a target system based on a pool of control policies, the controller being configured to:
 receive the pool of control policies, the pool of control policies comprising a plurality of control policies;   receive weights for weighting each control policy of the plurality of control policies;   weight the plurality of control policies by the weights to provide a weighted aggregated control policy;   control the target system using the weighted aggregated control policy;   receive performance data relating to a performance of the controlled target system;   adjust the weights by the processor based on the received performance data to improve the performance of the controlled target system; and   reweight the plurality of control policies by the adjusted weights to adjust the weighted aggregated control policy.   
     
     
         15 . In a non-transitory computer-readable storage medium that stores instructions executable by one or more processors to control a target system based on a pool of control policies, the instructions comprising:
 receiving the pool of control policies, the pool of control policies comprising a plurality of control policies;   receiving weights for weighting each control policy of the plurality of control policies;   weighting the plurality of control policies by the weights to provide a weighted aggregated control policy;   controlling the target system using the weighted aggregated control policy;   receiving performance data relating to a performance of the controlled target system;   adjusting the weights by the processor based on the received performance data to improve the performance of the controlled target system; and   reweighting the plurality of control policies by the adjusted weights to adjust the weighted aggregated control policy.   
     
     
         16 . The non-transitory computer-readable storage medium of  claim 15 , wherein adjusting the weights comprises training a neural network run by the processor. 
     
     
         17 . The non-transitory computer-readable storage medium of  claim 16 , wherein the instructions further comprise:
 receiving operational data of at least one source system; and   calculating the plurality of control policies from different data sets of the operational data.   
     
     
         18 . The non-transitory computer-readable storage medium of  claim 17 , wherein calculating the plurality of control policies comprises training the neural network or a further neural network. 
     
     
         19 . The non-transitory computer-readable storage medium of  claim 17 , wherein calculating the plurality of control policies comprises using a reward function relating to a performance of the at least on source system, and
 wherein adjusting the weights comprises using the reward function for the adjusting of the weights.   
     
     
         20 . The non-transitory computer-readable storage medium of  claim 15 , wherein the performance data comprises state data relating to a current state of the target system, and
 wherein the weighting of the plurality of control policies, the reweighting of the plurality of control policies, or the weighting of the plurality of control policies and the reweighting of the plurality of control policies depends on the state data.