Deep module and fitting module system and method for motion-based lane detection with multiple sensors

Abstract

A method of lane detection for a non-transitory computer readable storage medium storing one or more programs is disclosed. The one or more programs include instructions, which when executed by a computing device, cause the computing device to perform the following steps comprising: generating a ground truth; off-line training a lane detection algorithm by using the ground truth, the lane detection algorithm using parameters that express a lane marking in an arc; on-line generating a predicted lane marking; comparing the predicted lane marking against the ground truth; and off-line refining the lane detection algorithm.

What is claimed is: 
     
       1. A method of lane detection for a non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a computing device, cause the computing device to perform the following steps comprising:
 generating a ground truth; 
 off-line training a lane detection algorithm by using the ground truth, the lane detection algorithm using parameters that express a lane marking in an arc; 
 on-line generating a predicted lane marking for a current view; 
 comparing the predicted lane marking against the ground truth; and 
 off-line refining the lane detection algorithm by using the lane template associated with the current view to generate an improved lane template used for improved lane detection of a next view; 
 wherein, a detected lane map is generated relative to a moving vehicle using the improved lane template and the predicted lane marking. 
 
     
     
       2. The method according to  claim 1 , wherein generating a ground truth includes:
 collecting data in an environment by using sensors that include an inertial measurement unit (IMU) module, a global positioning system (GPS) module and a mapping (MAP) module; and 
 generating a labeled lane marking by annotating a lane marking expressed in god's view. 
 
     
     
       3. The method according to  claim 2 , wherein the IMU module provides information on at least one of a vehicle pose or a vehicle speed, and the global positioning system (GPS) module provides global position information. 
     
     
       4. The method according to  claim 2 , wherein the MAP module is configured to create a map of its surroundings, and orient a vehicle itself within this map. 
     
     
       5. The method according to  claim 1 , wherein on-line generating a predicted lane marking includes:
 generating a hit-map image for the current view based on the lane detection algorithm; and 
 generating a fitted lane marking based on the hit-map image and a lane template that includes features of a view immediately previous to the current view. 
 
     
     
       6. The method according to  claim 5 , wherein generating a fitted lane marking includes:
 optimizing, based on priors or constraints, the lane template to obtain a local optimal. 
 
     
     
       7. The method according to  claim 5  further comprising:
 determining that a confidence level of the fitted lane marking is reasonable, using the parameters; and 
 outputting the fitted lane marking as a predicted lane marking. 
 
     
     
       8. The method according to  claim 5  further comprising:
 determining that a confidence level of the fitted lane marking is unreasonable, using the parameters; and 
 rejecting the fitted lane marking. 
 
     
     
       9. The method according to  claim 1 , wherein off-line refining the lane detection algorithm includes:
 adding additional ground truth data in the off-line training. 
 
     
     
       10. The method according to  claim 1  further comprising:
 on-line generating another predicted lane marking, using a refined lane detection algorithm. 
 
     
     
       11. A system for lane detection, the system comprising:
 an internet server, comprising:
 an I/O port, configured to transmit and receive electrical signals to and from a client device; 
 a memory; 
 one or more processing units; and 
 one or more programs stored in the memory and configured for execution by the one or more processing units, the one or more programs including instructions for:
 generating a ground truth; 
 off-line training a lane detection algorithm by using the ground truth, the lane detection algorithm using parameters that express a lane marking in an arc; 
 on-line generating a predicted lane marking for a current view; 
 comparing the predicted lane marking against the ground truth; and 
 off-line refining the lane detection algorithm by using the lane template associated with the current view to generate an improved lane template used for improved lane detection of a next view; 
 
 
 wherein, a detected lane map is generated relative to a moving vehicle using the improved lane template and the predicted lane marking. 
 
     
     
       12. The system according to  claim 11 , wherein generating a ground truth includes:
 collecting data in an environment by using sensors that include an inertial measurement unit (IMU) module, a global positioning system (GPS) module and a mapping (MAP) module; and 
 generating a labeled lane marking by annotating a lane marking expressed in god's view. 
 
     
     
       13. The system according to  claim 12 , wherein the IMU module provides information on at least one of a vehicle pose or a vehicle speed, and the global positioning system (GPS) module provides global position information. 
     
     
       14. The system according to  claim 12 , wherein the MAP module is configured to create a map of its surroundings, and orient a vehicle itself within this map. 
     
     
       15. The system according to  claim 11 , wherein on-line generating a predicted lane marking includes:
 generating a hit-map image for the current view based on the lane detection algorithm; and 
 generating a fitted lane marking based on the hit-map image and a lane template that includes features of a view immediately previous to the current view. 
 
     
     
       16. The system according to  claim 15 , wherein generating a fitted lane marking includes:
 optimizing, based on priors or constraints, the lane template to obtain a local optimal. 
 
     
     
       17. The system according to  claim 15  further comprising:
 determining that a confidence level of the fitted lane is reasonable, using the parameters; and 
 outputting the fitted lane marking as a predicted lane marking. 
 
     
     
       18. The system according to  claim 15  further comprising:
 determining that a confidence level of the fitted lane is unreasonable, using the parameters; and 
 rejecting the fitted lane marking. 
 
     
     
       19. The system according to  claim 11 , wherein off-line refining the lane detection algorithm includes:
 adding additional ground truth data in the off-line training. 
 
     
     
       20. The system according to  claim 11  further comprising:
 on-line generating another predicted lane marking, using a refined lane detection algorithm.