Potential collision warning system based on road user intent prediction

Abstract

An apparatus comprising a memory to store an observed trajectory of a pedestrian, the observed trajectory comprising a plurality of observed locations of the pedestrian over a first plurality of timesteps; and a processor to generate a predicted trajectory of the pedestrian, the predicted trajectory comprising a plurality of predicted locations of the pedestrian over the first plurality of timesteps and over a second plurality of timesteps occurring after the first plurality of timesteps; determine a likelihood of the predicted trajectory based on a comparison of the plurality of predicted locations of the pedestrian over the first plurality of timesteps and the plurality of observed locations of the pedestrian over the first plurality of timesteps; and responsive to the determined likelihood of the predicted trajectory, provide information associated with the predicted trajectory to a vehicle to warn the vehicle of a potential collision with the pedestrian.

1 .- 20 . (canceled) 
     
     
         21 . At least one non-transitory machine-readable storage medium comprising instructions, which when executed by processor circuitry of a vehicle computing system, cause the processor circuitry to perform operations to:
 obtain data of an observed state of a pedestrian, the pedestrian observed from a vehicle in an environment, and the observed state of the pedestrian comprising a history of a plurality of observed positions of the pedestrian over a plurality of past time steps;   obtain data of an observed state of the vehicle, the observed state of the vehicle comprising a history of a plurality of observed positions of the vehicle over the plurality of past time steps;   generate a predicted pedestrian trajectory based on the history of the plurality of observed positions of the pedestrian over the plurality of past time steps, the predicted pedestrian trajectory generated over a plurality of future time steps occurring after the plurality of past time steps;   generate a predicted vehicle trajectory based on the history of the plurality of observed positions of the vehicle over the plurality of past time steps, the predicted vehicle trajectory generated over the plurality of future time steps;   determine a likelihood of the predicted pedestrian trajectory over the plurality of future time steps; and   utilize the determined likelihood of the predicted pedestrian trajectory and the predicted vehicle trajectory over the plurality of future time steps to change operation of the vehicle on a roadway within the environment.   
     
     
         22 . The machine-readable storage medium of  claim 21 , wherein the predicted vehicle trajectory is generated based on at least one of: a state of a traffic light of an intersection in the environment, or road information of the environment. 
     
     
         23 . The machine-readable storage medium of  claim 21 , wherein the instructions further cause the processor circuitry to:
 determine a likelihood of the predicted vehicle trajectory over the plurality of future time steps;   wherein the operations to utilize the predicted vehicle trajectory include operations to utilize the determined likelihood of the predicted vehicle trajectory.   
     
     
         24 . The machine-readable storage medium of  claim 21 , wherein the observed state of the pedestrian further comprises a velocity and an acceleration of the pedestrian. 
     
     
         25 . The machine-readable storage medium of  claim 21 , wherein the predicted vehicle trajectory is represented by respective positions of the vehicle at each of the plurality of future time steps. 
     
     
         26 . The machine-readable storage medium of  claim 21 , wherein the instructions further cause the processor circuitry to:
 provide a command to the vehicle to control the operation of the vehicle.   
     
     
         27 . The machine-readable storage medium of  claim 26  wherein the command relates to steering or speed of the vehicle. 
     
     
         28 . The machine-readable storage medium of  claim 21 , wherein the operations to generate the predicted pedestrian trajectory and to generate the predicted vehicle trajectory are performed by a trained neural network model. 
     
     
         29 . The machine-readable storage medium of  claim 21 , wherein the data of the observed state of the pedestrian is based on a plurality of images captured by a camera. 
     
     
         30 . A computing device, comprising:
 memory configured to store:
 data of an observed state of a pedestrian, the pedestrian observed from a vehicle in an environment, and the observed state of the pedestrian comprising a history of a plurality of observed positions of the pedestrian over a plurality of past time steps; and 
 data of an observed state of the vehicle, the observed state of the vehicle comprising a history of a plurality of observed positions of the vehicle over the plurality of past time steps; and 
   processor circuitry configured to:
 generate a predicted pedestrian trajectory based on the history of the plurality of observed positions of the pedestrian over the plurality of past time steps, the predicted pedestrian trajectory generated over a plurality of future time steps occurring after the plurality of past time steps; 
 generate a predicted vehicle trajectory based on the history of the plurality of observed positions of the vehicle over the plurality of past time steps, the predicted vehicle trajectory generated over the plurality of future time steps; 
 determine a likelihood of the predicted pedestrian trajectory over the plurality of future time steps; and 
 utilize the determined likelihood of the predicted pedestrian trajectory and the predicted vehicle trajectory over the plurality of future time steps to change operation of the vehicle on a roadway within the environment. 
   
     
     
         31 . The computing device of  claim 30 , wherein the predicted vehicle trajectory is generated based on at least one of: a state of a traffic light of an intersection in the environment, or road information of the environment. 
     
     
         32 . The computing device of  claim 30 , wherein the processor circuitry is further configured to:
 determine a likelihood of the predicted vehicle trajectory over the plurality of future time steps;   wherein to utilize the predicted vehicle trajectory includes to utilize the determined likelihood of the predicted vehicle trajectory.   
     
     
         33 . The computing device of  claim 30 , wherein the observed state of the pedestrian further comprises a velocity and an acceleration of the pedestrian. 
     
     
         34 . The computing device of  claim 30 , wherein the predicted vehicle trajectory is represented by respective positions of the vehicle at each of the plurality of future time steps. 
     
     
         35 . The computing device of  claim 30 , wherein the processor circuitry is further configured to:
 provide a command to the vehicle to control the operation of the vehicle.   
     
     
         36 . The computing device of  claim 35 , wherein the command relates to steering or speed of the vehicle. 
     
     
         37 . The computing device of  claim 30 , wherein operations to generate the predicted pedestrian trajectory and to generate the predicted vehicle trajectory are performed by a trained neural network model. 
     
     
         38 . The computing device of  claim 30 , wherein the data of the observed state of the pedestrian is based on a plurality of images captured by a camera. 
     
     
         39 . An apparatus, comprising:
 means for receiving data from (i) an observed state of a pedestrian, the pedestrian observed from a vehicle in an environment, and the observed state of the pedestrian comprising a history of a plurality of observed positions of the pedestrian over a plurality of past time steps, and (ii) an observed state of the vehicle, the observed state of the vehicle comprising a history of a plurality of observed positions of the vehicle over the plurality of past time steps;   means for generating (i) a predicted pedestrian trajectory based on the history of the plurality of observed positions of the pedestrian over the plurality of past time steps, the predicted pedestrian trajectory generated over a plurality of future time steps occurring after the plurality of past time steps, and (ii) a predicted vehicle trajectory based on the history of the plurality of observed positions of the vehicle over the plurality of past time steps, the predicted vehicle trajectory generated over the plurality of future time steps;   means for determining (i) a likelihood of the predicted pedestrian trajectory over the plurality of future time steps; and   means for causing a change in operation of the vehicle on a roadway within the environment, based on the determined likelihood of the predicted pedestrian trajectory and the predicted vehicle trajectory over the plurality of future time steps.   
     
     
         40 . The apparatus of  claim 39 , further comprising:
 means for transmitting a command to the vehicle to control the operation of the vehicle.   
     
     
         41 . The apparatus of  claim 39 , further comprising:
 means for implementing a trained neural network model, for generating the predicted pedestrian trajectory and for generating the predicted vehicle trajectory.   
     
     
         42 . The apparatus of  claim 39 , further comprising:
 means for capturing a plurality of images, to provide the data for the observed state of the pedestrian.