Position determination of a vehicle using image segmentations

Abstract

A method of determining a position of a vehicle in which a reference map is provided. The reference map comprises segmentations of a reference image with landmarks. A measurement image of a vehicle environment is captured and segmentations of the measurement image and neighborhood graphs are determined to obtain a measurement map, wherein a segmentation is represented by a vertex and where a neighborhood graph comprises the vertex and edges containing information to identify neighboring vertices of the vertex. Segmentations of the reference image are compared with the segmentations represented by the vertices of the measurement image and the neighborhood graphs, and segmentations contained in the reference image and in measurement image are determined. The vehicle's position is estimated with reference to the reference map during its movement along a road based on a result of the comparison.

1 . A method of determining a position of a vehicle, the method comprising:
 capturing a measurement image of a vehicle environment;   determining segmentations represented by a vertex of the measurement image and neighborhood graphs, wherein the neighborhood graph comprises the vertex and edges containing information to identify neighboring vertices of the vertex, to obtain a measurement map;   comparing segmentations of a reference map comprising segmentations of a reference image with landmarks to the segmentations represented by the vertices of the measurement image and the neighborhood graphs;   determining segmentations contained in the reference map and in the measurement image; and   estimating the position of the vehicle based on the reference map during movement of the vehicle based on a result of the comparison.   
     
     
         2 . The method according to  claim 1 , wherein comparing the segmentations of the reference map to the segmentations represented by the vertices of the measurement image and the neighborhood graphs comprises performing a rough localization to determine a road partition on which the vehicle moves and selecting a reference image from the reference map related to the road partition. 
     
     
         3 . The method according to  claim 1 , wherein comparing the segmentations of the reference map with the segmentations represented by the vertices of the measurement image and the neighborhood graphs comprises selecting an object from a set of objects contained in an image, and
 wherein a segmentation represents an object.   
     
     
         4 . The method according to  claim 1 , wherein comparing the segmentations of the reference image with the segmentations represented by the vertices of the measurement image and the neighborhood graphs comprises:
 capturing real-time LiDAR data of the vehicle environment;   extracting features from a segmentation of a selected object by a first Resnet;   extracting features from the LiDAR data of the selected object by a first PointNet; and   providing the extracted features to a common PointNet.   
     
     
         5 . The method according to  claim 4 , wherein comparing the segmentations of the reference map with the segmentations represented by the vertices of the measurement image and the neighborhood graphs comprises:
 extracting features from neighbor segmentations of the segmentation of the selected object and providing the extracted features to a GAT;   extracting features from LiDAR points cloud of a neighboring object of the selected object and providing the extracted features to the GAT;   describing the extracted features containing spatial information; and   providing the described reference image features to the common PointNet.   
     
     
         6 . The method according to  claim 5 , further comprising:
 concatenating the extracted features from the first ResNet, the first PointNet and the GAT; and   determining a similarity between the features of the object of the reference image and the features of the object of the measurement image.   
     
     
         7 . The method according to  claim 6 , wherein determining the similarity between the features of the object of the landmark map network part and the features of the object of the measurement map part comprises predicting labels by a Multi-Layer Perceptron (MLP) and calculating a loss function by calculating a cross entropy. 
     
     
         8 . The method according to  claim 1 , wherein providing the reference map comprises:
 capturing LIDAR data points and a reference image along a road for a road partition;   mapping the LIDAR data points to the reference image;   determining objects on the reference image and determining landmark segmentations from the reference image using a semantic segmentation neural network; and   constructing a graph topological landmark map containing vertices corresponding each to a segmentation and edges, wherein an edge identifies neighboring vertices of a vertex.   
     
     
         9 - 10 . (canceled)