Apparatus for autonomous driving and method and system for calibrating sensor thereof

Abstract

The autonomous driving device including a communication circuit configured to communicate with an unmanned aerial vehicle, a plurality of sensors disposed in the autonomous vehicle to monitor all directions of the autonomous vehicle, and a processor, wherein the processor is configured to: control the unmanned aerial vehicle to hover at each of a plurality of waypoints of a designated flight path by controlling a relative position of the unmanned aerial vehicle through the communication circuit, change a posture angle of the unmanned aerial vehicle to a plurality of posture angles corresponding to the waypoints of the flight path, generate a plurality of images including the checkerboard and corresponding to the plurality of waypoints and the plurality of posture angles through the plurality of sensors, and calibrate the plurality of sensors on the basis of a relationship between matching points of the plurality of images.

What is claimed is: 
     
       1. An autonomous driving device included in an autonomous vehicle, the autonomous driving device comprising:
 a communication circuit configured to communicate with an unmanned aerial vehicle to which a checkerboard is fixed; 
 a plurality of sensors disposed in the autonomous vehicle to monitor all directions of the autonomous vehicle; and 
 a processor, 
 wherein the processor is configured to:
 control the unmanned aerial vehicle to hover at each of a plurality of waypoints of a designated flight path by controlling a relative position of the unmanned aerial vehicle through the communication circuit; 
 change a posture angle of the unmanned aerial vehicle to a plurality of posture angles corresponding to the waypoints of the flight path; 
 generate a plurality of images including the checkerboard and corresponding to the plurality of waypoints and the plurality of posture angles through the plurality of sensors; and 
 calibrate the plurality of sensors on the basis of a relationship between matching points of the plurality of images. 
 
 
     
     
       2. The autonomous driving device of  claim 1 , wherein the processor designates the flight path including the plurality of waypoints so that the unmanned aerial vehicle is positioned in a visible range of at least one of the plurality of sensors. 
     
     
       3. The autonomous driving device of  claim 2 , wherein the processor designates the plurality of waypoints to include all regions in which visible ranges of two of the plurality of sensors overlap each other. 
     
     
       4. The autonomous driving device of  claim 2 , wherein the processor designates, as the plurality of waypoints, intersections between a line spaced a specific distance from the autonomous vehicle and points obtained by dividing a circumference of the autonomous vehicle by first unit angles. 
     
     
       5. The autonomous driving device of  claim 2 , wherein the processor is configured to:
 designate the plurality of posture angles related to the waypoints so that the checkerboard and the autonomous vehicle form a plurality of angles ranging from a first critical angle to a second critical angle at the waypoints; and 
 associate the designated posture angles with the waypoints of the flight path. 
 
     
     
       6. The autonomous driving device of  claim 2 , wherein
 the processor stores the designated flight path in a memory when the flight path is designated, and 
 the processor uses the designated flight path during subsequent calibration of the plurality of sensors. 
 
     
     
       7. The autonomous driving device of  claim 1 , wherein the processor updates the flight path on the basis of a result of the calibration of the plurality of sensors. 
     
     
       8. The autonomous driving device of  claim 1 , wherein the processor is configured to:
 determine an intrinsic parameter and an extrinsic parameter during the calibration of the plurality of sensors; and 
 determine position coordinates of a nearby object on the basis of the determined intrinsic parameter and the extrinsic parameter when the nearby object is detected through the plurality of sensors. 
 
     
     
       9. A sensor calibration system comprising:
 an unmanned aerial vehicle to which a checkerboard is fixed; and 
 an autonomous driving device configured to control the unmanned aerial vehicle and included in an autonomous vehicle, 
 wherein the autonomous driving device is configured to:
 control the unmanned aerial vehicle to hover at each of a plurality of waypoints of a designated flight path by controlling a relative position of the unmanned aerial vehicle; 
 change a posture angle of the unmanned aerial vehicle to a plurality of posture angles corresponding to the waypoints of the flight path; 
 generate a plurality of images including the checkerboard and corresponding to the plurality of waypoints and the plurality of posture angles through a plurality of sensors; and 
 calibrate the plurality of sensors on the basis of a relationship between matching points of the plurality of images. 
 
 
     
     
       10. The sensor calibration system of  claim 9 , wherein
 the unmanned aerial vehicle is fixed or housed in a region of the autonomous vehicle, and 
 the unmanned aerial vehicle is configured to start flight corresponding to the flight path according to a command of the autonomous driving device. 
 
     
     
       11. The sensor calibration system of  claim 9 , wherein
 the unmanned aerial vehicle changes a flight state, which is at least one of a flight position or a posture angle, according to a command received from the autonomous driving device, and 
 the unmanned aerial vehicle transmits flight state reporting information corresponding to a change of the at least one flight state to the autonomous driving device. 
 
     
     
       12. The sensor calibration system of  claim 9 , wherein
 the unmanned aerial vehicle comprises a Global Positioning System (GPS) module configured to detect current position coordinates of the unmanned aerial vehicle, and 
 the unmanned aerial vehicle controls a flight position according to a command received from the autonomous vehicle on the basis of the detected current position coordinates. 
 
     
     
       13. The sensor calibration system of  claim 9 , wherein the autonomous driving device is configured to:
 designate the flight path including the plurality of waypoints so that the unmanned aerial vehicle is positioned in a visible range of at least one of the plurality of sensors; and 
 control a flight position of the unmanned aerial vehicle according to the designated flight path. 
 
     
     
       14. The sensor calibration system of  claim 9 , wherein the autonomous driving vehicle is configured to:
 designate the plurality of posture angles related to the waypoints so that the checkerboard and the autonomous vehicle form a plurality of angles ranging from a first critical angle to a second critical angle at the waypoints, associate the designated posture angles with the waypoints of the flight path; and 
 control a posture angle of the unmanned aerial vehicle to be the plurality of posture angles at the waypoints. 
 
     
     
       15. A sensor calibration method performed by an autonomous driving device, the sensor calibration method comprising:
 controlling an unmanned aerial vehicle, to which a checkerboard is fixed, to hover at each of a plurality of waypoints of a designated flight path; 
 changing a posture angle of the unmanned aerial vehicle to a plurality of posture angles corresponding to the waypoints of the flight path; 
 generating a plurality of images including the checkerboard and corresponding to the plurality of waypoints and the plurality of posture angles through a plurality of sensors included in the autonomous driving device; and 
 calibrating the plurality of sensors on the basis of a relationship between matching points of the plurality of images. 
 
     
     
       16. The sensor calibration method of  claim 15 , further comprising designating the flight path including the plurality of waypoints so that the unmanned aerial vehicle is positioned in a visible range of at least one of the plurality of sensors. 
     
     
       17. The sensor calibration method of  claim 16 , where the designating of the flight path comprises designating the plurality of waypoints to include all regions in which visible ranges of two of the plurality of sensors overlap each other. 
     
     
       18. The sensor calibration method of  claim 16 , wherein the designating of the flight path comprises designating, as the plurality of waypoints, intersections between a curve spaced a specific distance from an autonomous vehicle and points obtained by dividing a circumference of the autonomous vehicle by first unit angles. 
     
     
       19. The sensor calibration method of  claim 16 , wherein the designating of the flight path comprises designating the plurality of posture angles related to the waypoints so that the checkerboard and an autonomous vehicle form a plurality of angles ranging from a first critical angle to a second critical angle at the waypoints. 
     
     
       20. The autonomous driving device of  claim 15 , further comprising updating the flight path on the basis of a result of the calibration of the plurality of sensors.