Boom sprayer including machine feedback control

Abstract

A boom sprayer includes any number of components to treat plants as the boom sprayer travels through a plant field. The components take actions to treat plants or facilitate treating plants. The boom sprayer includes any number of sensors to measure the state of the boom sprayer as the boom sprayer treats plants. The boom sprayer includes a control system to generate actions for the components to treat plants in the field. The control system includes an agent executing a model that functions to improve the performance of the boom sprayer treating plants. Performance improvement can be measured by the sensors of the boom sprayer. The model is an artificial neural network that receives measurements as inputs and generates actions that improve performance as outputs. The artificial neural network is trained using actor-critic reinforcement learning techniques.

What is claimed is: 
     
       1. A method for controlling a plurality of actuation controllers of a plurality of components of a boom sprayer to treat plants as the boom sprayer travels through a plant field, the method comprising:
 determining a state vector comprising a plurality of state elements, each of the state elements representing a measurement of a state of a subset of the plurality of components of the boom sprayer, and each of the plurality of components controlled by an actuation controller communicatively coupled to a computer mounted on the boom sprayer; 
 inputting, using the computer, the state vector into a control model to generate an action vector comprising a plurality of action elements for the boom sprayer, each of the action elements specifying an action to be taken by the boom sprayer in the plant field, and the actions, in aggregate, predicted to optimize one or more performance metrics of the boom sprayer; and 
 actuating a subset of the plurality of actuation controllers to execute the actions in the plant field based on the action vector, the subset of actuation controllers changing a configuration of the subset of components such that the state of the boom sprayer changes, and wherein actuating the subset of actuation controllers comprises:
 determining a set of machine instructions in each actuation controller of the subset such that the machine instructions change the configuration of each component when received by the actuation controller, 
 accessing a data network communicatively coupling the actuation controllers, and 
 sending the set of machine instructions to each actuation controller of the subset via the data network. 
 
 
     
     
       2. The method of  claim 1 , wherein the control model comprises a function representing a relationship between the state vector received as an input to the control model and the action vector generated as an output to the control model, and the function is a model trained using reinforcement learning to reward actions that improve treatments applied to a plant in the plant field by the boom sprayer. 
     
     
       3. The method of  claim 1  wherein the control model comprises an artificial neural network comprising:
 a plurality of neural nodes including a set of input nodes for receiving an input to the artificial neural network and a set of output nodes for outputting an output to the artificial neural network, where
 each neural node represents a sub-function for determining an output for the artificial neural network from the input of the artificial neural network, and 
 each input node is connected to one or more output nodes by a connection of a plurality of weighted connections; and 
 
 a function configured to generate actions for the boom sprayer which improve the boom sprayer performance, the function defined by a plurality of sub-functions and weighted connections of the artificial neural network. 
 
     
     
       4. The method of  claim 3 , wherein:
 each state element of the state vector is connected to one or more input nodes by a connection of the plurality of weighted connections, 
 each action element of the action vector is connected to one or more output nodes by a connection of the plurality of weighted connections, and 
 the function is configured to generate action elements of the action vector from state elements of the state vector. 
 
     
     
       5. The method of  claim 3 , wherein the artificial neural network is a first artificial neural network from a pair of similarly configured artificial neural networks acting as an actor-critic pair and used to train the first artificial neural network to generate actions that improve the boom sprayer performance. 
     
     
       6. The method of  claim 5 , wherein:
 the first artificial neural network inputs state vectors and values for the weighted connections and outputs action vectors, the values for the weighted connections modifying the function for generating actions for the boom sprayer that improve boom sprayer performance, and 
 a second neural network inputting a reward vector and a state vector and outputting the values for the weighted connections, the reward vector comprising elements signifying improvement in performance of the boom sprayer from a previously executed action that improves boom sprayer performance. 
 
     
     
       7. The method of  claim 6  wherein the elements of the reward vector are determined using at least one measurement quantifying capabilities of a subset the components of the boom sprayer that were previously actuated based on the previously executed action. 
     
     
       8. The method of  claim 5 , wherein an operator of the boom sprayer can select one or more metrics for performance improvement, the metrics including any of a distance between the boom sprayer and a plant in the plant field, a distance between the boom sprayer and the plant, a distance between the boom sprayer and a ground surface, a distance between the boom sprayer and the ground surface over time, an amount of plant treated, and a quality of treatment applied to plant. 
     
     
       9. The method of  claim 5 , wherein the state vector is obtained from plurality of boom sprayers taking a plurality of actions from a plurality of action vectors to treat plants in the plant field. 
     
     
       10. The method of  claim 5 , wherein the state vectors and action vectors are simulated from a set of state vectors obtained from a plurality of boom sprayers taking a set of actions from a seed set of action vectors to treat plants in the plant field. 
     
     
       11. The method of  claim 1 , wherein determining the state vector comprises:
 accessing a data network communicatively coupling a plurality of sensors, each sensor for providing a measurement quantifying capabilities of a subset of the components of the boom sprayer; and 
 determining elements of the state vector based on the measurements included in the data network. 
 
     
     
       12. The method of  claim 11 , wherein the plurality of sensors can include any of an ultrasonic sensor, tilt sensor, roll angle sensor, GPS sensor, vehicle wheel speed sensor, steering angle sensor, tread width sensors, suspension sensors, and an IMU sensors. 
     
     
       13. The method of  claim 1 , wherein the plurality of state elements comprise any of:
 a frame height representing a height of the boom sprayer relative to a ground of the plant field; 
 a frame angle representing an angle of the boom sprayer frame relative to a direction of gravity; 
 a sprayer potential representing a measure of an electric potential of the boom sprayer; 
 a sprayer position representing a position of the boom sprayer in a coordinate system; 
 a suspension height representing a distance between the suspension of the boom sprayer and the ground; and 
 a sprayer motion representing a set of motion sensing information characterizing the boom sprayer. 
 
     
     
       14. The method of  claim 1 , wherein the action elements can specify actions including any of:
 adjusting a position of a left frame relative to a ground of the plant field or a center frame of the boom sprayer; 
 adjusting a position of the center frame of the boom sprayer relative to the ground or a fixed center frame of the boom sprayer; and 
 adjusting a position of a right frame of the boom sprayer relative to the ground or the center frame. 
 
     
     
       15. The method of  claim 1 , wherein the plurality of components of the boom sprayer can include any of a fixed or floating center frame, a center boom frame, a left boom, and a right boom, wherein the fixed or floating center frame supports a spray boom assembly comprising a plurality of spray nozzles for applying treatment to a plant in the plant field. 
     
     
       16. The method of  claim 1 , wherein the components of the boom sprayer are configured to treat plants including any of corn, wheat, or rice. 
     
     
       17. The method of  claim 1 , wherein action elements of the action vector are a numerical representation of the action. 
     
     
       18. The method of  claim 1 , wherein state elements of the state vector are a numerical representation of measurements making up the state vector. 
     
     
       19. The method of  claim 1 , wherein the control model comprises a reinforcement learning model, where the reinforcement learning model implements one or more of the following:
 an action-value function; 
 a state-value function; 
 policy iteration; 
 value iteration; 
 temporal-difference learning; 
 Q-learning; 
 Value prediction; and 
 Actor-critic training. 
 
     
     
       20. The method of  claim 1 , wherein the control model implements one of the following:
 A linear quadratic (LQR) extension; and 
 A long short-term memory (LSTM) model interleaved with a proximal policy optimization (PPO). 
 
     
     
       21. The method of  claim 1 , wherein the control model is trained using Gaussian Process dynamics. 
     
     
       22. A non-transitory computer readable storage medium storing instructions for controlling a plurality of actuation controllers of a plurality of components of a boom sprayer to treat plants encoded thereon that, when executed by one or more processors, cause the one or more processors to perform the steps including:
 determining a state vector comprising a plurality of state elements, each of the state elements representing a measurement of a state of a subset of the plurality of components of the boom sprayer, and each of the plurality of components controlled by an actuation controller communicatively coupled to a computer mounted on the boom sprayer; 
 inputting, using the computer, the state vector into a control model to generate an action vector comprising a plurality of action elements for the boom sprayer, each of the action elements specifying an action to be taken by the boom sprayer in the plant field, and the actions, in aggregate, predicted to optimize one or more performance metrics of the boom sprayer; and 
 actuating a subset of the plurality of actuation controllers to execute the actions in the plant field based on the action vector, the subset of actuation controllers changing a configuration of the subset of components such that the state of the boom sprayer changes, and wherein actuating the subset of actuation controllers comprises:
 determining a set of machine instructions in each actuation controller of the subset such that the machine instructions change the configuration of each component when received by the actuation controller, 
 accessing a data network communicatively coupling the actuation controllers; and 
 sending the set of machine instructions to each actuation controller of the subset via the data network. 
 
 
     
     
       23. A boom sprayer comprising:
 one or more spray mechanisms; 
 one or more actuation controllers communicatively coupled to the one or more spray mechanisms and for controlling the one or more spray mechanisms; 
 one or more computer processors; and 
 a computer-readable storage medium storing instructions that when executed causes one or more processors to:
 determine a state vector comprising a plurality of state elements, each of the state elements representing a measurement of a state of a subset of the plurality of components of the boom sprayer, and each of the plurality of components controlled by an actuation controller communicatively coupled to a computer mounted on the boom sprayer; 
 input, using the one or more computer processors, the state vector into a control model to generate an action vector comprising a plurality of action elements for the boom sprayer, each of the action elements specifying an action to be taken by the boom sprayer in the plant field, and the actions, in aggregate, predicted to optimize one or more performance metrics of the boom sprayer; and 
 actuate a subset of the plurality of actuation controllers to execute the actions in the plant field based on the action vector, the subset of actuation controllers changing a configuration of the subset of components such that the state of the boom sprayer changes, and wherein actuating the subset of actuation controllers comprises:
 determining a set of machine instructions in each actuation controller of the subset such that the machine instructions change the configuration of each component when received by the actuation controller, 
 accessing a data network communicatively coupling the actuation controllers; and 
 sending the set of machine instructions to each actuation controller of the subset via the data network.