Autonomous vehicle controlled based upon a lidar data segmentation system

Abstract

An autonomous vehicle is described herein. The autonomous vehicle includes a lidar sensor system. The autonomous vehicle additionally includes a computing system that executes a lidar segmentation system, wherein the lidar segmentation system is configured to identify objects that are in proximity to the autonomous vehicle based upon output of the lidar sensor system. The computing system further includes a deep neural network (DNN), where the lidar segmentation system identifies the objects in proximity to the autonomous vehicle based upon output of the DNN.

What is claimed is: 
     
         1 . An autonomous vehicle (AV) comprising:
 an engine;   a braking system;   a steering system;   a lidar sensor; and   a computing system that is in communication with the engine, the braking system, the steering system, and the lidar sensor, wherein the computing system comprises:
 a processor; and 
 memory that stores instructions that, when executed by the processor, cause the processor to perform acts comprising:
 receiving lidar data, the lidar data based upon output of the lidar sensor, the lidar data comprising a plurality of points representative of positions of objects in a driving environment of the AV; 
 assigning a label to a first point in the points that indicates that the first point is representative of ground cover or vegetation based upon output of a deep neural network (DNN) that is configured to classify points as being representative of ground cover or vegetation; 
 generating a segmentation of the lidar data based upon the label being assigned to the first point, wherein the segmentation is indicative of a second point in the lidar data and a third point in the lidar data being representative of a same object; and 
 controlling at least one of the engine, the braking system, or the steering system during operation of the AV in the driving environment based upon the segmentation. 
 
   
     
     
         2 . The AV of  claim 1 , wherein the output of the DNN comprises a probability that the first point is representative of vegetation. 
     
     
         3 . The AV of  claim 2 , wherein the label indicates that the first point is representative of vegetation, wherein assigning the label is based upon the probability exceeding a threshold value. 
     
     
         4 . The AV of  claim 1 , wherein responsive to receipt of input features pertaining to the first point, the output of the DNN comprises:
 a first probability that the first point is representative of vegetation;   a second probability that the first point is representative of ground cover; and   a third probability that the first point is representative of an object of a type other than vegetation or ground cover.   
     
     
         5 . The AV of  claim 1 , wherein the output of the DNN comprises a probability that the first point is representative of ground cover. 
     
     
         6 . The AV of  claim 1 , wherein generating the segmentation of the lidar data comprises assigning group labels to the points in the lidar data, each group label indicating one of a plurality of groups of points, each group of points representative of a different respective object in the driving environment. 
     
     
         7 . The AV of  claim 6 , wherein generating the segmentation comprises assigning a same first group label to the first point and a fourth point based upon the first point and the fourth point being labeled as representative of vegetation, the first group label indicative of first group that is representative of a vegetation object in the driving environment. 
     
     
         8 . The AV of  claim 6 , wherein generating the segmentation comprises assigning different group labels to the first point and a fourth point based upon the first point being labeled as representative of vegetation and the fourth point not being labeled as representative of vegetation. 
     
     
         9 . The AV of  claim 1 , wherein generating the segmentation comprises excluding the first point from consideration by a segmentation algorithm based upon the label being assigned to the first point. 
     
     
         10 . The AV of  claim 1 , the acts further comprising assigning a respective label to each of a first group of points in the points based upon output of the DNN, the labels assigned to the first group of points indicating that the first group of points are representative of vegetation or ground cover in the driving environment, wherein generating the segmentation is based further upon the labels assigned to the first group of points. 
     
     
         11 . The AV of  claim 10 , wherein generating the segmentation comprises excluding the first point and the first group of points from consideration by a segmentation algorithm based upon the labels being assigned to the first point and the first group of points. 
     
     
         12 . A method for controlling operation of an autonomous vehicle (AV), comprising:
 receiving lidar data from a lidar sensor system of the AV, the lidar data based upon output of at least one lidar sensor, the lidar data comprising a plurality of points representative of positions of objects in a driving environment of the AV;   assigning a label to a first point in the points based upon output of a deep neural network (DNN) that is configured to output a probability that a point in lidar data is representative of ground cover or vegetation, the label indicating that the first point is representative of ground cover or vegetation in the driving environment;   generating a segmentation of the lidar data based upon the label being assigned to the first point, wherein the segmentation is indicative of a second point in the lidar data and a third point in the lidar data being representative of a same object; and   controlling, based upon the segmentation, at least one of an engine of the AV, a braking system of the AV, or a steering system of the AV during operation of the AV in the driving environment.   
     
     
         13 . The method of  claim 12 , wherein the output of the DNN comprises a probability that the first point is representative of vegetation. 
     
     
         14 . The method of  claim 12 , wherein the output of the DNN comprises a probability that the first point is representative of ground cover. 
     
     
         15 . The method of  claim 14 , wherein the label indicates that the first point is representative of ground cover, wherein assigning the label is based upon the probability exceeding a threshold value. 
     
     
         16 . The method of  claim 12 , wherein generating the segmentation of the lidar data comprises assigning group labels to the points in the lidar data, each group label indicating one of a plurality of groups of points, each group of points representative of a different respective object in the driving environment. 
     
     
         17 . The method of  claim 12 , wherein generating the segmentation comprises executing a lidar segmentation algorithm over the lidar data based upon the label being assigned to the first point. 
     
     
         18 . The method of  claim 12 , the acts further comprising assigning a respective label to each of a first group of points in the points based upon output of the DNN, the labels assigned to the first group of points indicating that the first group of points are representative of vegetation or ground cover in the driving environment, wherein generating the segmentation is based further upon the labels assigned to the first group of points. 
     
     
         19 . The method of  claim 18 , wherein generating the segmentation comprises excluding the first point and the first group of points from consideration by a segmentation algorithm based upon the labels being assigned to the first point and the first group of points. 
     
     
         20 . An autonomous vehicle (AV) comprising:
 a computer-readable storage medium comprising instructions that, when executed by a processor, cause the processor to perform acts comprising:
 receiving a lidar point cloud from a lidar sensor system of the AV, the lidar point cloud based upon output of at least one lidar sensor, the lidar point cloud comprising a plurality of points representative of positions of objects in a driving environment of the AV; 
 assigning a label to a first point in the points that indicates that the first point is representative of vegetation in the driving environment based upon output of a deep neural network (DNN), wherein the DNN is trained to receive features pertaining to the first point as input and to output a probability that the first point is representative of vegetation in the driving environment, the label assigned to the first point based upon the probability exceeding a threshold probability; 
 generating a segmentation of the lidar point cloud based upon the label being assigned to the first point, wherein the segmentation indicates that a second point in the points and a third point in the points are representative of a same object; and 
 controlling, based upon the second point and the third point being indicated as representative of the same object, at least one of an engine of the AV, a braking system of the AV, or a steering system of the AV during operation of the AV in the driving environment.