End-To-End Interpretable Motion Planner for Autonomous Vehicles

Abstract

Systems and methods for generating motion plans including target trajectories for autonomous vehicles are provided. An autonomous vehicle may include or access a machine-learned motion planning model including a backbone network configured to generate a cost volume including data indicative of a cost associated with future locations of the autonomous vehicle. The cost volume can be generated from raw sensor data as part of motion planning for the autonomous vehicle. The backbone network can generate intermediate representations associated with object detections and objection predictions. The motion planning model can include a trajectory generator configured to evaluate one or more potential trajectories for the autonomous vehicle and to select a target trajectory based at least in part on the cost volume generate by the backbone network.

1 - 20 . (canceled) 
     
     
         21 . A computer-implemented method, comprising:
 obtaining sensor data associated with an environment surrounding an autonomous vehicle;   generating a plurality of trajectory proposals for the autonomous vehicle, the plurality of trajectory proposals respectively corresponding to a plurality of potential paths of the autonomous vehicle through the environment;   determining a respective score for a respective trajectory proposal of the plurality of trajectory proposals by:
 obtaining one or more cost values associated with a respective potential path corresponding to the respective trajectory proposal, wherein:
 the one or more cost values are obtained from cost data descriptive of costs for a plurality of positions in the environment along the respective potential path, and 
 the cost data is generated based at least in part on the sensor data by a machine-learned cost model component; 
 
   selecting a target trajectory from the one or more trajectory proposals based at least in part on the respective scores for the one or more trajectory proposals; and   controlling a motion of the autonomous vehicle based at least in part on the selected target trajectory.   
     
     
         22 . The computer-implemented method of  claim 21 , wherein:
 the cost data describes costs for possible positions that the autonomous vehicle can take within a planning horizon;   the respective potential path describes a plurality of proposed positions that the autonomous vehicle can take at a plurality of timesteps within the planning horizon; and   the method comprises:
 obtaining, from the cost data, the cost values associated with the proposed positions at the plurality of timesteps. 
   
     
     
         23 . The computer-implemented method of  claim 22 , wherein the cost data is generated by extracting features from the sensor data and processing the features with the machine-learned cost model component. 
     
     
         24 . The computer-implemented method of  claim 21 , comprising:
 generating, using a backbone machine-learned model component, feature data based at least in part on the sensor data;   processing, using a machine-learned forecasting model component, the feature data to generate forecasting data indicating one or more forecasted positions of an object in the environment; and   processing, using the machine-learned cost model component, the feature data to generate the cost data.   
     
     
         25 . The computer-implemented method of  claim 24 , wherein the feature data is based at least in part on map data processed by the backbone machine-learned model component. 
     
     
         26 . The computer-implemented method of  claim 22 , wherein the cost data comprises a cost volume having dimensions associated with a region of interest that comprises the plurality of positions in the environment. 
     
     
         27 . The computer-implemented method of  claim 26 , wherein the cost volume comprises a temporal dimension comprising the plurality of timesteps. 
     
     
         28 . The computer-implemented method of  claim 21 , wherein generating the one or more trajectory proposals for the autonomous vehicle comprises:
 sampling a set of physically possible trajectories for the autonomous vehicle.   
     
     
         29 . The computer-implemented method of  claim 28 , wherein the sampling comprises:
 sampling a shape of a curve;   sampling a motion parameter comprising at least one of: a velocity parameter or an acceleration parameter; and   combining the sampled shape and the sampled motion parameter to obtain a respective possible trajectory.   
     
     
         30 . An autonomous vehicle computing system for controlling an autonomous vehicle, the autonomous vehicle computing system comprising:
 one or more processors; and   one or more non-transitory computer-readable media that store instructions that are executable by the one or more processors to cause the autonomous vehicle computing system to perform operations, the operations comprising:
 obtaining sensor data associated with an environment surrounding an autonomous vehicle; 
 generating a plurality of trajectory proposals for the autonomous vehicle, the plurality of trajectory proposals respectively corresponding to a plurality of potential paths of the autonomous vehicle through the environment; 
 determining a respective score for a respective trajectory proposal of the plurality of trajectory proposals by:
 obtaining one or more cost values associated with a respective potential path corresponding to the respective trajectory proposal, wherein:
 the one or more cost values are obtained from cost data descriptive of costs for a plurality of positions in the environment along the respective potential path, and 
 the cost data is generated based at least in part on the sensor data by a machine-learned cost model component; 
 
 
 selecting a target trajectory from the one or more trajectory proposals based at least in part on the respective scores for the one or more trajectory proposals; and 
 controlling a motion of the autonomous vehicle based at least in part on the selected target trajectory. 
   
     
     
         31 . The autonomous vehicle of  claim 30 , wherein:
 the cost data describes costs for possible positions that the autonomous vehicle can take within a planning horizon;   the respective potential path describes a plurality of proposed positions that the autonomous vehicle can take at a plurality of timesteps within the planning horizon; and   the operations comprise:
 obtaining, from the cost data, the cost values associated with the proposed positions at the plurality of timesteps. 
   
     
     
         32 . The autonomous vehicle of  claim 31 , wherein the cost data is generated by extracting features from the sensor data and processing the features with the machine-learned cost model component. 
     
     
         33 . The autonomous vehicle of  claim 30 , wherein the operations comprise:
 generating, using a backbone machine-learned model component, feature data based at least in part on the sensor data;   processing, using a machine-learned forecasting model component, the feature data to generate forecasting data indicating one or more forecasted positions of an object in the environment; and   processing, using the machine-learned cost model component, the feature data to generate the cost data.   
     
     
         34 . The autonomous vehicle of  claim 33 , wherein the feature data is based at least in part on map data processed by the backbone machine-learned model component. 
     
     
         35 . The autonomous vehicle of  claim 31 , wherein the cost data comprises a cost volume having dimensions associated with a region of interest that comprises the plurality of positions in the environment. 
     
     
         36 . The autonomous vehicle of  claim 35 , wherein the cost volume comprises a temporal dimension comprising the plurality of timesteps. 
     
     
         37 . The autonomous vehicle of  claim 30 , wherein generating the one or more trajectory proposals for the autonomous vehicle comprises:
 sampling a set of physically possible trajectories for the autonomous vehicle.   
     
     
         38 . The autonomous vehicle of  claim 37 , wherein the sampling comprises:
 sampling a shape of a curve;   sampling a motion parameter comprising at least one of: a velocity parameter or an acceleration parameter; and   combining the sampled shape and the sampled motion parameter to obtain a respective possible trajectory.   
     
     
         39 . One or more non-transitory computer-readable media that store instructions that are executable by one or more processors to cause a computing system to perform operations, the operations comprising:
 obtaining sensor data associated with an environment surrounding an autonomous vehicle;   generating a plurality of trajectory proposals for the autonomous vehicle, the plurality of trajectory proposals respectively corresponding to a plurality of potential paths of the autonomous vehicle through the environment;   determining a respective score for a respective trajectory proposal of the plurality of trajectory proposals by:
 obtaining one or more cost values associated with a respective potential path corresponding to the respective trajectory proposal, wherein:
 the one or more cost values are obtained from cost data descriptive of costs for a plurality of positions in the environment along the respective potential path, and 
 the cost data is generated based at least in part on the sensor data by a machine-learned cost model component; 
 
   selecting a target trajectory from the one or more trajectory proposals based at least in part on the respective scores for the one or more trajectory proposals; and   controlling a motion of the autonomous vehicle based at least in part on the selected target trajectory.   
     
     
         40 . The one or more non-transitory computer-readable media of  claim 39 , wherein:
 the cost data describes costs for possible positions that the autonomous vehicle can take within a planning horizon;   the respective potential path describes a plurality of proposed positions that the autonomous vehicle can take at a plurality of timesteps within the planning horizon; and   the operations comprise:
 obtaining, from the cost data, the cost values associated with the proposed positions at the plurality of timesteps.