Calibration of laser and vision sensors

Abstract

Automatic calibration between laser and vision sensors carried by a mobile platform, and associated systems and methods are disclosed herein. A representative method includes evaluating depth-based feature points obtained from the laser sensor with edge information obtained from the vision sensor and generating calibration rules based thereon.

1 - 106 . (canceled) 
     
     
         107 . A computer-implemented method for generating a point cloud, the method comprising:
 obtaining observation data generated by at least one vision sensor within a time period;   evaluating states associated with a laser unit at different points in time within the time period based at least on the observation data;   determining one or more transformation rules based at least on the states associated with the laser unit for transforming between one or more reference systems and a target reference system each of which being associated with the laser unit, wherein the one or more reference systems are associated with the laser unit at the different points in time within the time period and the target reference system is associated with the laser unit at a target point in time within the time period; and   generating the point cloud by transforming data obtained by the laser unit to the target reference system based at least on the one or more transformation rules, the data obtained by the laser unit corresponding to the different points in time within the time period.   
     
     
         108 . The method of  claim 107 , wherein determining the one or more transformation rules further comprises:
 computing transformation matrices for the laser unit at the different points in time with respect to the target point in time, wherein each transformation matrix is computed using a corresponding state associated with the laser unit at a corresponding point in time.   
     
     
         109 . The method of  claim 108 , wherein transforming data obtained by the laser unit based at least on the one or more transformation rules to the target reference system further comprises:
 transforming the data obtained by the laser unit at the corresponding point in time to the target point in time using a corresponding transformation matrix.   
     
     
         110 . The method of  claim 107 , wherein the at least one vision sensor and the laser unit are carried by a mobile platform. 
     
     
         111 . The method of  claim 107 , wherein the at least one vision sensor comprises at least one of a stereo camera or a monocular camera. 
     
     
         112 . The method of  claim 107 , wherein obtaining the observation data comprises obtaining the observation data at different data acquisition rates from at least two different vision sensors. 
     
     
         113 . The method of  claim 107 , wherein the laser unit has a different data acquisition rate than the at least one vision sensor. 
     
     
         114 . The method of  claim 107 , wherein the states associated with the laser unit are evaluated based on states associated with the at least one vision sensor, or wherein the states associated with the laser unit include at least one of a position, a speed, or a rotation of the laser unit. 
     
     
         115 . The method of  claim 107 , further comprising selecting one or more feature points from the point cloud based at least on one or more depth differences between points within the point cloud. 
     
     
         116 . The method of  claim 115 , wherein selecting the one or more feature points from the point cloud is further based on a relationship between the one or more depth differences and a threshold discontinuity in depth measurement. 
     
     
         117 . The method of  claim 115 , further comprising evaluating the selected feature points, using edge information obtained from the at least one vision sensor based at least on a target function, the target function defined at least by positions of the selected feature points when projected to a reference system associated with the at least one vision sensor. 
     
     
         118 . The method of  claim 117 , further comprising:
 generating at least one calibration rule for calibration between the laser unit and the at least one vision sensor based at least on evaluating the selected feature points using the edge information; and   causing the calibration between the laser unit and the at least one vision sensor using the at least one calibration rule.   
     
     
         119 . The method of  claim 107 , further comprising:
 converting an image obtained from the at least one vision sensor into a grayscale image; and   determining edge information based at least on a difference between at least one pixel of the grayscale image and one or more pixels within a threshold proximity of the at least one pixel.   
     
     
         120 . The method of  claim 107 , wherein the one or more transformation rules are at least partially defined in accordance with a position and an orientation of the at least one vision sensor relative to a mobile platform. 
     
     
         121 . The method of  claim 107 , wherein the method further comprises:
 selecting one or more feature points from the point cloud; and   evaluating the selected feature points, using edge information obtained from the at least one vision sensor.   
     
     
         122 . The method of  claim 121 , wherein the method further comprises generating at least one calibration rule for calibration between the laser unit and the at least one vision sensor based at least on evaluating the selected feature points using the edge information. 
     
     
         123 . A non-transitory computer-readable medium storing computer-executable instructions that, when executed, cause one or more processors associated with a mobile platform to perform operations, the operations comprising:
 obtaining observation data generated by at least one vision sensor within to a time period;   evaluating states associated with a laser unit at different points in time within the time period based at least on the observation data;   determining one or more transformation rules based at least on the states associated with the laser unit for transforming between one or more reference systems and a target reference system each of which being associated with the laser unit, wherein the one or more reference systems are associated with the laser unit at the different points in time within the time period, and the target reference system is associated with the laser unit at a target point in time within the time period; and   generating the point cloud by transforming data obtained by the laser unit to the target reference system based at least on the one or more transformation rules, the data obtained by the laser unit corresponding to the different points in time within the time period.   
     
     
         124 . The computer-readable medium of  claim 123 , wherein the operations further comprise:
 selecting one or more feature points from the point cloud; and   evaluating the selected feature points, using edge information obtained from the at least one vision sensor based at least on a target function, the target function defined at least by positions of the selected feature points when projected to a reference system associated with the at least one vision sensor.   
     
     
         125 . A apparatus including a programmed controller that at least partially controls one or more motions of the apparatus, wherein the programmed controller includes one or more processors to perform operations, the operations comprising:
 obtaining observation data generated by at least one vision sensor within a time period;   evaluating states associated with a laser unit at different points in time within the time period based at least on the observation data;   determining one or more transformation rules based at least on the states associated with the laser unit for transforming between one or more reference systems and a target reference system each of which being associated with the laser unit, wherein the one or more reference systems are associated with the laser unit at the different points in time within the time period and the target reference system is associated with the laser unit at a target point in time within the time period; and   generating the point cloud by transforming data obtained by the laser unit to the target reference system based at least on the one or more transformation rules, the data obtained by the laser unit corresponding to the different points in time within the time period.   
     
     
         126 . The apparatus of  claim 125 , being coupled to a vehicle.