Predicting articulated object states

Abstract

A vehicle computing system may implement techniques to determine whether two objects in an environment are related as an articulated object. The techniques may include applying heuristics and algorithms to object representations (e.g., bounding boxes) to determine whether two objects are related as a single object with two portions that articulate relative to each other. The techniques may include predicting future states of the articulated object in the environment. One or more model(s) may be used to determine presence of the articulated object and/or predict motion of the articulated object in the future. Based on the presence and/or motion of the articulated object, the vehicle computing system may control operation of the vehicle.

What is claimed is: 
     
       1. A system comprising:
 one or more processors; and 
 one or more non-transitory computer-readable media storing instructions executable by the one or more processors, wherein the instructions, when executed, cause the system to perform operations comprising:
 receiving sensor data from a sensor associated with a vehicle in an environment; 
 determining, based at least in part on the sensor data, presence of an articulated object in the environment, the articulated object including a first portion and a second portion; 
 inputting, into a model, state data associated with the first portion of the articulated object at a first time; 
 determining, by the model and based at least in part on the state data, a covariant relationship between the first portion and the second portion of the articulated object, the covariant relationship identifying a covariance of the state data for determining a predicted state of the second portion; 
 determining, by the model and based at least in part on the covariance identified in the covariant relationship, a predicted state of the second portion of the articulated object at a second time after the first time; and 
 controlling the vehicle in the environment based at least in part on the predicted state of the articulated object. 
 
 
     
     
       2. The system of  claim 1 , the operations further comprising:
 applying, by the model, a Kalman filter algorithm to the state data to determine the covariant relationship between the first portion and the second portion. 
 
     
     
       3. The system of  claim 2 , wherein the Kalman filter algorithm is a derivative free Kalman filter algorithm. 
     
     
       4. The system of  claim 1 , wherein:
 the state data is associated with the first portion and comprises one or more of: position data, orientation data, heading data, velocity data, speed data, acceleration data, yaw data, yaw rate data, distance data indicating a distance from an edge of the first portion to an intersection point between the first portion and the second portion, or turning rate data associated with the articulated object, and 
 the predicted state of the second portion is determined based at least in part on the covariance between a first value of the first portion and a second value of the second portion. 
 
     
     
       5. The system of  claim 1 , wherein:
 the first portion is a front portion of the articulated object relative to a direction of travel, 
 the second portion is a rear portion of the articulated object relative to the direction of travel, 
 the predicted state includes position data, yaw data, or velocity data, and 
 the covariance is between a first point in the first portion and a second point in the second portion. 
 
     
     
       6. A method comprising:
 detecting an articulated object in an environment, the articulated object including a first portion and a second portion; 
 inputting first state data associated with the first portion of the articulated object into a model at a first time; 
 determining, by the model and based at least in part on the first state data, a relationship between the first portion and the second portion of the articulated object; 
 receiving, as an output from the model and based at least in part on the relationship, a predicted state of the second portion of the articulated object at a second time after the first time, the predicted state of the second portion being determined independent of the model receiving second state data for the second portion after the first time; and 
 controlling a vehicle in the environment based at least in part on predicted state of the articulated object. 
 
     
     
       7. The method of  claim 6 , further comprising:
 applying, by the model, a filtering algorithm to the state data to determine the relationship between the first portion and the second portion, 
 wherein the output by the model is based at least in part on the filtering algorithm. 
 
     
     
       8. The method of  claim 7 , wherein the filtering algorithm is a derivative free Kalman filter algorithm. 
     
     
       9. The method of  claim 6 , wherein the state data is associated with at least one of the first portion or the second portion and comprises one or more of: position data, orientation data, heading data, velocity data, speed data, acceleration data, yaw data, yaw rate data, distance data indicating a distance from an edge of the first portion or the second portion to an intersection point between the first portion and the second portion, or turning rate data associated with the articulated object. 
     
     
       10. The method of  claim 6 , wherein:
 the first portion is a front portion of the articulated object relative to a direction of travel, 
 the second portion is a rear portion of the articulated object relative to the direction of travel, 
 the model identifies a covariance between a first point in the first portion and a second point in the second portion, 
 the predicted state of the second portion identifies a position, a yaw, or a velocity of the second portion based at least in part on the covariance. 
 
     
     
       11. The method of  claim 6 , further comprising:
 receiving sensor data from one or more sensors associated with the vehicle in the environment; and 
 updating, based at least in part on the sensor data, the relationship between the first portion and the second portion of the articulated object. 
 
     
     
       12. The method of  claim 6 , further comprising:
 determining an offset value between a first distance, a first velocity, or a first yaw associated with the first portion and a second distance, a second velocity, or a second yaw associated with the second portion of the articulated object, 
 wherein the output from the model identifying the predicted state of the first portion and the second portion is based at least in part on the offset value. 
 
     
     
       13. The method of  claim 12 , wherein the relationship comprises a velocity covariance, a yaw covariance, or a distance covariance between the first portion and the second portion. 
     
     
       14. The method of  claim 6 , further comprising:
 determining a first velocity of the first portion or a second velocity of the second portion, 
 wherein the output from the model identifying the predicted state of the second portion is based at least in part on the first velocity or the second velocity. 
 
     
     
       15. The method of  claim 6 , further comprising:
 determining a direction of travel of the articulated object; 
 determining, based at least in part on the direction of travel, the first portion or the second portion as a front portion, 
 wherein the output from the model identifying the predicted state of the front portion. 
 
     
     
       16. The method of  claim 6 , further comprising:
 receiving first sensor data from a first sensor and second sensor data from a second sensor different from the first sensor, the first sensor and the second sensor associated with the vehicle in the environment; and 
 determining a joint point between the first portion and the second portion based at least in part on the first sensor data and the second sensor data, 
 wherein the output from the model identifying the predicted state of the articulated object is based at least in part on the joint point. 
 
     
     
       17. One or more non-transitory computer-readable media storing instructions executable by one or more processors, wherein the instructions, when executed, cause the one or more processors to perform operations comprising:
 detecting an articulated object in an environment, the articulated object including a first portion and a second portion; 
 inputting state data associated with the articulated object into a model; 
 determining, by the model and based at least in part on the state data, a relationship between the first portion and the second portion of the articulated object; 
 receiving, as an output from the model and based at least in part on the relationship, a predicted state of the second portion of the articulated object at a future time independent of the model processing additional state data of the second portion; and 
 controlling a vehicle in the environment based at least in part on predicted state of the articulated object. 
 
     
     
       18. The one or more non-transitory computer-readable media of  claim 17 , the operations further comprising:
 applying, by the model, a filtering algorithm to the state data to determine the relationship between the first portion and the second portion, 
 wherein the output by the model is based at least in part on the filtering algorithm. 
 
     
     
       19. The one or more non-transitory computer-readable media of  claim 18 , wherein the filtering algorithm is an unscented Kalman filter algorithm. 
     
     
       20. The one or more non-transitory computer-readable media of  claim 17 , wherein:
 the state data is associated with the first portion and comprises one or more of: position data, orientation data, heading data, velocity data, speed data, acceleration data, yaw rate data, or turning rate data associated with the articulated object, and 
 the predicted state of the second portion is determined based at least in part on the covariance between a first value of the first portion and a second value of the second portion.