Pre-tracking sensor event detection and fusion

Abstract

This application discloses a computing system to implement pre-tracking sensor event detection and fusion in an assisted or automated driving system of a vehicle. The computing system can receive an environmental model including sensor measurement data from different types of sensors in the vehicle. The computing system can identify, on a per-sensor type basis, patterns in the sensor measurement data indicative of possible objects proximate to the vehicle. The computing system can associate the patterns in the sensor measurement data from different types of the sensors to identify detection events corresponding to the possible objects proximate to the vehicle. The computing system also can generate values and confidence levels corresponding to properties of the detection events. The computing system can utilize the detection events and corresponding values and confidence levels to pre-classify, identify, and track objects in the environment model.

1 . A method comprising:
 receiving, by a computing system, sensor measurement data from different types of sensors in a vehicle, wherein the sensor measurement data is spatially and temporally aligned in an environmental model associated with the vehicle;   identifying, by the computing system on a per-sensor type basis, patterns in the sensor measurement data indicative of possible objects proximate to the vehicle; and   associating, by the computing system, the patterns in the sensor measurement data from different types of the sensors to identify detection events corresponding to the possible objects proximate to the vehicle, wherein a control system for the vehicle is configured to control operation of the vehicle based, at least in part, on the detection events.   
     
     
         2 . The method of  claim 1 , wherein identifying the patterns in the sensor measurement data further comprises extracting features from image data corresponding to at least one image capture device. 
     
     
         3 . The method of  claim 1 , wherein identifying patterns in the sensor measurement data further comprises identifying, on the per-sensor type basis, one or more clusters of data points in the sensor measurement data. 
     
     
         4 . The method of  claim 3 , wherein identifying the clusters of the data points in the sensor measurement data further comprises:
 identifying spatial locations in a particular time period for the data points in the sensor measurement data from the environmental model;   determining a state corresponding to the data points in the sensor measurement data based, at least in part, on inter-frame differences corresponding to the data points in the sensor measurement data; and   grouping a subset of the data points in the sensor measurement data into one of the clusters based on the identifying spatial locations in the particular time period and the determined state corresponding to the data points.   
     
     
         5 . The method of  claim 1 , further comprising determining, by the computing system, inter-frame differences in the sensor measurement data corresponding the patterns in the sensor measurement data, wherein associating the patterns in the sensor measurement data from different types of the sensors to identify the detection events corresponding to the possible objects proximate to the vehicle is based, at least in part, on the inter-frame differences in the sensor measurement data. 
     
     
         6 . The method of  claim 1 , wherein the detection events have properties associated with the possible objects proximate to the vehicle, and wherein associating the patterns in the sensor measurement data from different types of the sensors to identify the detection events further comprises generating confidence levels corresponding to the properties of the detection events. 
     
     
         7 . The method of  claim 6 , wherein the properties include at least one a unity, a velocity, an orientation, a center of gravity, an existence, a size, or a novelty associated with the detection events. 
     
     
         8 . An apparatus comprising at least one memory device storing instructions configured to cause one or more processing devices to perform operations comprising:
 receiving sensor measurement data from different types of sensors in a vehicle, wherein the sensor measurement data is spatially and temporally aligned in an environmental model associated with the vehicle;   identifying, on a per-sensor type basis, patterns in the sensor measurement data indicative of possible objects proximate to the vehicle; and   associating the patterns in the sensor measurement data from different types of the sensors to identify detection events corresponding to the possible objects proximate to the vehicle, wherein a control system for the vehicle is configured to control operation of the vehicle based, at least in part, on the detection events.   
     
     
         9 . The apparatus of  claim 8 , wherein identifying the patterns in the sensor measurement data further comprises extracting features from image data corresponding to at least one image capture device. 
     
     
         10 . The apparatus of  claim 8 , wherein identifying patterns in the sensor measurement data further comprises identifying, on the per-sensor type basis, one or more clusters of data points in the sensor measurement data. 
     
     
         11 . The apparatus of  claim 10 , wherein identifying the clusters of the data points in the sensor measurement data further comprising:
 identifying spatial locations in a particular time period for the data points in the sensor measurement data from the environmental model;   determining a state corresponding to the data points in the sensor measurement data based, at least in part, on inter-frame differences corresponding to the data points in the sensor measurement data; and   grouping a subset of the data points in the sensor measurement data into one of the clusters based on the identifying spatial locations in the particular time period and the determined state corresponding to the data points.   
     
     
         12 . The apparatus of  claim 8 , wherein the instructions are further configured to cause the one or more processing devices to perform operations comprising determining inter-frame differences in the sensor measurement data corresponding the patterns in the sensor measurement data, wherein associating the patterns in the sensor measurement data from different types of the sensors to identify the detection events corresponding to the possible objects proximate to the vehicle is based, at least in part, on the inter-frame differences in the sensor measurement data. 
     
     
         13 . The apparatus of  claim 8 , wherein the detection events have properties associated with the possible objects proximate to the vehicle, and wherein associating the patterns in the sensor measurement data from different types of the sensors to identify the detection events further comprises generating confidence levels corresponding to the properties of the detection events. 
     
     
         14 . The apparatus of  claim 13 , wherein the properties include at least one a unity, a velocity, an orientation, a center of gravity, an existence, a size, or a novelty associated with the detection event. 
     
     
         15 . A system comprising:
 a memory device configured to store machine-readable instructions; and   a computing system including one or more processing devices, in response to executing the machine-readable instructions, configured to:
 receive sensor measurement data from different types of sensors in a vehicle, wherein the sensor measurement data is spatially and temporally aligned in an environmental model associated with the vehicle; 
 identify, on a per-sensor type basis, patterns in the sensor measurement data indicative of possible objects proximate to the vehicle; and 
 associate the patterns in the sensor measurement data from different types of the sensors to identify detection events corresponding to the possible objects proximate to the vehicle, wherein a control system for the vehicle is configured to control operation of the vehicle based, at least in part, on the detection events. 
   
     
     
         16 . The system of  claim 15 , wherein the one or more processing devices, in response to executing the machine-readable instructions, are configured to identify patterns in the sensor measurement data by extracting features from image data corresponding to at least one image capture device. 
     
     
         17 . The system of  claim 15 , wherein the one or more processing devices, in response to executing the machine-readable instructions, are configured to identify patterns in the sensor measurement data by identifying, on the per-sensor type basis, one or more clusters of data points in the sensor measurement data. 
     
     
         18 . The system of  claim 17 , wherein the one or more processing devices, in response to executing the machine-readable instructions, are configured to identify the clusters of the data points in the sensor measurement data by:
 identifying spatial locations in a particular time period for the data points in the sensor measurement data from the environmental model;   determining a state corresponding to the data points in the sensor measurement data based, at least in part, on inter-frame differences corresponding to the data points in the sensor measurement data; and   grouping a subset of the data points in the sensor measurement data into one of the clusters based on the identifying spatial locations in the particular time period and the determined state corresponding to the data points.   
     
     
         19 . The system of  claim 15 , wherein the one or more processing devices, in response to executing the machine-readable instructions, are configured to:
 determine inter-frame differences in the sensor measurement data corresponding the patterns in the sensor measurement data; and   associate the patterns in the sensor measurement data from different types of the sensors to identify the detection events corresponding to the possible objects proximate to the vehicle based, at least in part, on the inter-frame differences in the sensor measurement data.   
     
     
         20 . The system of  claim 15 , wherein the one or more processing devices, in response to executing the machine-readable instructions, are configured to generate confidence levels corresponding to properties of the detection events based, at least in part, on the association of the patterns in the sensor measurement data from different types of the sensors.