Apparatus for acquiring 3-dimensional maps of a scene

Abstract

An active sensor for performing active measurements of a scene is presented. The active sensor includes at least one transmitter configured to emit light pulses toward at least one target object in the scene, wherein the at least one target object is recognized in an image acquired by a passive sensor; at least one receiver configured to detect light pulses reflected from the at least one target object; a controller configured to control an energy level, a direction, and a timing of each light pulse emitted by the transmitter, wherein the controller is further configured to control at least the direction for detecting each of the reflected light pulses; and a distance measurement circuit configured to measure a distance to each of the at least one target object based on the emitted light pulses and the detected light pulses.

What is claimed is: 
     
       1. A system, comprising:
 a. a LIDAR sensor mounted on a vehicle, including:
 i. a transmitter configured to emit light pulses toward a scene, 
 ii. a receiver configured to detect reflected light pulses from the scene, the LIDAR sensor configured to generate in response to the reflected light pulses a LIDAR scene representation including LIDAR data conveying a plurality of distance values, 
 
 b. an image sensor mounted on the vehicle to generate an image representation including image data conveying a plurality of pixels, 
 c. a computer system receiving the LIDAR data and the image data and configured for:
 i. fusing the LIDAR scene representation and the image scene representation to produce fused data providing a 3D scene map, the fusing including associating the distance values of the LIDAR data to the pixels of the image data, wherein for a first plurality of pixels in the image data, distance values of the LIDAR data are available and for a second plurality of pixels in the image data, no distance values of the LIDAR data are available, 
 ii. for the second plurality of pixels in the image data for which no distance values are available in the LIDAR data estimating distance values at least in part on a basis of distance values in the LIDAR data available for the first plurality of pixels in the image data that are similar in color to the second plurality of pixels, 
 iii. determining a proximity of a target object within the scene to the vehicle, and 
 iv. controlling an energy level of light pulses emitted by the transmitter, wherein the energy level is one of at least two available non-zero energy levels and wherein the energy level of light pulses emitted by the transmitter is based at least in part on the determined proximity of the target object to the vehicle. 
 
 
     
     
       2. A system as defined in  claim 1 , wherein the LIDAR sensor and the image sensor are calibrated one respective to the other to provide a mapping between the LIDAR scene representation and the image scene representation wherein distance values of the LIDAR data are mapped to corresponding pixels of the image data, wherein the fused data providing the 3D scene map includes a plurality of data frames, each data frame including pixels associated with distance measurements according to the mapping. 
     
     
       3. A system as defined in  claim 2 , wherein the controlling of the energy level including decreasing an energy level with an increasing proximity of the target object to the vehicle. 
     
     
       4. A system as defined in  claim 3 , wherein the controlling of the energy level including increasing the energy level with a decreasing proximity of the target object to the vehicle. 
     
     
       5. A system as defined in  claim 4 , wherein the computer system is configured to segment the image data into a plurality of segments. 
     
     
       6. A system as defined in  claim 5 , wherein the computer system is configured to define segments in the image data based at least in part on color similarity of pixels in a segment. 
     
     
       7. A system as defined in  claim 6 , wherein the computer system identifies three or more distance values in the LIDAR data associated with pixels in the particular segment and estimates on the basis of the plurality of distance values by using a plane equation, distance values for other pixels in the particular segment for which no distance values are available in the LIDAR data. 
     
     
       8. A system as defined in  claim 1 , wherein the computer system is configured to perform an object detection process in the image data to identify a target object in the image scene representation. 
     
     
       9. A system as defined in  claim 8 , wherein the object detection process includes classifying the target object in an object category selected among a plurality of object categories. 
     
     
       10. A system as defined in  claim 9 , wherein the plurality of object categories includes an object category associated with static objects. 
     
     
       11. A system as defined in  claim 9 , wherein the plurality of object categories includes an object category associated with moving objects. 
     
     
       12. A system as defined in  claim 9 , wherein the plurality of object categories includes a suspicious object category and a non-suspicious object category, the computer system classifying a target object in the non-suspicious objects category when a size of the target object is such that the vehicle can ride over the target object safely. 
     
     
       13. An automated method, comprising:
 a. providing a LIDAR sensor mounted on a vehicle including:
 i. a transmitter configured to emit light pulses toward a scene, 
 ii. a receiver configured to detect reflected light pulses from the scene, the LIDAR sensor configured to generate in response to the reflected light pulses a LIDAR scene representation including LIDAR data conveying distance values, 
 
 b. providing an image sensor mounted on the vehicle to generate an image scene representation including image data conveying a plurality of pixels, 
 c. providing a computer system for receiving the LIDAR data and the image data, and:
 i. fusing the LIDAR scene representation and the image scene representation to produce fused data providing a 3D scene map, the fusing including associating the distance values of the LIDAR data to the pixels of the image data, wherein for a first plurality of pixels in the image data, distance values in the LIDAR data are available and wherein for a second plurality of pixels in the image data, no distance values of the LIDAR data are available, 
 ii. for the second plurality of pixels in the image data for which no distance values are available in the LIDAR data estimating distance values at least in part on a basis of distance values in the LIDAR data available for the first plurality of pixels in the image data that are similar in color to the second plurality of pixels, 
 iii. determining a proximity of a target object within the scene to the vehicle, 
 iv. controlling with the computer system an energy level of light pulses emitted by the transmitter, wherein the energy level is one of at least two available non-zero energy levels and wherein the energy level of light pulses emitted by the transmitter is based at least in part on the determined proximity of the target object to the vehicle. 
 
 
     
     
       14. A method as defined in  claim 13 , wherein the LIDAR sensor and the image sensor are calibrated one respective to the other to provide a mapping between the LIDAR scene representation and the image scene representation wherein distance values of the LIDAR data are mapped to corresponding pixels of the image data, wherein the fused data providing the 3D scene map includes a plurality of data frames, each data frame including pixels associated with distance measurements according to the mapping. 
     
     
       15. A method as defined in  claim 14 , wherein the controlling of the energy level including decreasing an energy level with an increasing proximity of the target object to the vehicle. 
     
     
       16. A method as defined in  claim 15 , wherein the controlling of the energy level including increasing the energy level with a decreasing proximity of the target object to the vehicle. 
     
     
       17. A method as defined in  claim 16 , wherein the computer system is configured to segment the image data into a plurality of segments. 
     
     
       18. A method as defined in  claim 17 , wherein the computer system is configured to define segments in the image data based at least in part on color similarity of pixels in a segment. 
     
     
       19. A method as defined in  claim 18 , wherein the computer system is configured to identify three or more distance values in the LIDAR data associated with pixels in the particular segment and estimates on the basis of the plurality of distance values by using a plane equation, distance values for other pixels in the particular segment for which no distance values are available in the LIDAR data.