Adjusting alignment for microwave transmissions based on an RL model

Abstract

It is provided a method for adjusting alignment for microwave transmissions from a microwave transmitter to a microwave receiver based on a reinforcement learning, RL, model. The method comprises the steps of: obtaining state space comprising external state space and internal state space, the external state space comprising at least one value of a parameter related to environmental conditions, and the internal state space relates to alignment of the microwave transmitter; determining an action in an action space, the action space comprising actions to adjust alignment of the microwave transmitter; obtaining a measurement of path loss for a transmission from the microwave transmitter to the microwave receiver; determining a reward value based on the path loss, wherein an increase in path loss results in a reduced reward value; and adjusting the RL model based on the obtained state space, the determined action and the determined reward value.

The invention claimed is: 
     
       1. A method for adjusting alignment for microwave transmissions from a microwave transmitter to a microwave receiver based on a reinforcement learning (RL) model, the method being performed in an alignment adjuster and comprising the steps of:
 obtaining state space comprising external state space and internal state space, the external state space comprising at least one value of a parameter related to environmental conditions at the microwave transmitter or microwave receiver, and the internal state space comprising at least one value of a parameter related to alignment of the microwave transmitter; 
 determining, based on the RL model and the state space, an action in an action space, the action space comprising actions to adjust alignment of the microwave transmitter; 
 obtaining a measurement of path loss for a transmission from the microwave transmitter to the microwave receiver; 
 determining a reward value based on the path loss, wherein an increase in path loss results in a reduced reward value; and 
 adjusting the RL model based on the obtained state space, the determined action and the determined reward value. 
 
     
     
       2. The method of  claim 1 , wherein the external state space is based also on captured images passed through a convolutional neural network model. 
     
     
       3. The method of  claim 1 , wherein the external state space comprises at least one measurement of the following parameters: ambient temperature, rate of precipitation, humidity, wind speed, wind direction, air pressure. 
     
     
       4. The method of  claim 1 , wherein the internal state space comprises transmission power of the microwave transmitter. 
     
     
       5. The method of  claim 4 , wherein the step of determining a reward value comprises determining the reward value also based on the transmission power, wherein, all else being equal, an increase in transmission power results in a reduced reward value. 
     
     
       6. The method of  claim 1 , wherein the internal state space comprises at least one of vertical angle of the microwave receiver and horizontal angle of the microwave receiver. 
     
     
       7. The method of  claim 1 , further comprising the step of:
 triggering the collection of additional parameters related to environmental conditions when a sequence of successive reward values indicate a performance less than a threshold performance. 
 
     
     
       8. An alignment adjuster for adjusting alignment for microwave transmissions from a microwave transmitter to a microwave receiver based on a reinforcement learning (RL) model, the alignment adjuster comprising:
 a processor; and 
 a memory wherein the alignment adjuster is configured to: 
 obtain state space comprising external state space and internal state space, the external state space comprising at least one value of a parameter related to environmental conditions at the microwave transmitter or microwave receiver, and the internal state space comprising at least one value of a parameter related to alignment of the microwave transmitter; 
 determine, based on the RL model and the state space, an action in an action space, the action space comprising actions to adjust alignment of the microwave transmitter; 
 obtain a measurement of path loss for a transmission from the microwave transmitter to the microwave receiver; 
 determine a reward value based on the path loss, wherein an increase in path loss results in a reduced reward value; and 
 adjust the RL model based on the obtained state space, the determined action and the determined reward value. 
 
     
     
       9. The alignment adjuster of  claim 8 , wherein the external state space is based also on captured images passed through a convolutional neural network model. 
     
     
       10. The alignment adjuster to of  claim 8 , wherein the external state space comprises at least one measurement of the following parameters: ambient temperature, rate of precipitation, humidity, wind speed, wind direction, air pressure. 
     
     
       11. The alignment adjuster of  claim 8 , wherein the internal state space comprises vertical angle of the microwave transmitter. 
     
     
       12. The alignment adjuster of  claim 8 , wherein the internal state space comprises horizontal angle of the microwave transmitter. 
     
     
       13. The alignment adjuster of  claim 8 , wherein the internal state space comprises transmission power of the microwave transmitter. 
     
     
       14. The alignment adjuster of  claim 13 , wherein the instructions to determine a reward value comprise instructions that, when executed by the processor, cause the alignment adjuster to determine the reward value also based on the transmission power, wherein, all else equal, an increase in transmission power results in a reduced reward value. 
     
     
       15. The alignment adjuster of  claim 8 , wherein the internal state space comprises at least one of vertical angle of the microwave receiver and horizontal angle of the microwave receiver. 
     
     
       16. The alignment adjuster of  claim 8 , further comprising instructions that, when executed by the processor, cause the alignment adjuster to trigger the collection of additional parameters related to environmental conditions when a sequence of successive reward values indicate a performance less than a threshold performance. 
     
     
       17. The alignment adjuster of  claim 16 , wherein the instructions triggering comprises triggering at least one unmanned aerial vehicle to collect environmental data between the microwave transmitter and the microwave receiver. 
     
     
       18. A non-transitory computer readable medium storing a computer program for adjusting alignment for microwave transmissions from a microwave transmitter to a microwave receiver based on a reinforcement learning (RL) model, the computer program comprising computer program code which, when run on an alignment adjuster, causes the alignment adjuster to:
 obtain state space comprising external state space and internal state space, the external state space comprising at least one value of a parameter related to environmental conditions at the microwave transmitter or microwave receiver, and the internal state space comprising at least one value of a parameter related to alignment of the microwave transmitter; 
 determine, based on the RL model and the state space, an action in an action space, the action space comprising actions to adjust alignment of the microwave transmitter; 
 obtain a measurement of path loss for a transmission from the microwave transmitter to the microwave receiver; 
 determine a reward value based on the path loss, wherein an increase in path loss results in a reduced reward value; and 
 adjust the RL model based on the obtained state space, the determined action and the determined reward value.