Time-of-Flight Sensor with Structured Light Illuminator

Abstract

The present disclosure relates to systems and methods that provide information about a scene based on a time-of-flight (ToF) sensor and a structured light pattern. In an example embodiment, a sensor system could include at least one ToF sensor configured to receive light from a scene. The sensor system could also include at least one light source configured to emit a structured light pattern and a controller that carries out operations. The operations include causing the at least one light source to illuminate at least a portion of the scene with the structured light pattern and causing the at least one ToF sensor to provide information indicative of a depth map of the scene based on the structured light pattern.

What is claimed is: 
     
         1 . A sensor system comprising:
 at least one time-of-flight (ToF) sensor configured to receive light from a scene;   at least one light source configured to emit a structured light pattern; and   a controller that carries out operations, the operations comprising:   dynamically adjusting the structured light pattern based on one or more retroreflective regions in the scene;   causing the at least one light source to illuminate at least a portion of the scene with the structured light pattern; and   causing the at least one ToF sensor to provide time of flight information indicative of a depth map of the scene based on the structured light pattern.   
     
     
         2 . The sensor system of  claim 1 , wherein dynamically adjusting the structured light pattern comprises lowering illumination levels for portions of the scene where the one or more retroreflector regions are present. 
     
     
         3 . The sensor system of  claim 1 , wherein the at least one ToF sensor comprises a plurality of complementary metal-oxide semiconductor (CMOS) or charge-coupled device (CCD) photosensitive elements. 
     
     
         4 . The sensor system of  claim 1 , wherein the structured light pattern comprises at least one of: a predetermined spatial distribution of light, a predetermined temporal distribution of light, or a predetermined spectral distribution of light. 
     
     
         5 . The sensor system of  claim 1 , wherein the structured light pattern comprises at least one of: a predetermined light pulse repetition rate, a predetermined light pulse duration, a predetermined light pulse intensity, or a predetermined light pulse duty cycle. 
     
     
         6 . The sensor system of  claim 1 , wherein the at least one light source comprises at least one of: a laser diode, a light-emitting diode, a plasma light source, a strobe light, a solid-state laser, or a fiber laser. 
     
     
         7 . The sensor system of  claim 1 , wherein dynamically adjusting the structured light pattern comprises selecting a desired structured light pattern from among a plurality of possible structured light patterns, wherein causing the at least one light source to illuminate at least a portion of the scene with the structured light pattern comprises illuminating the portion of the scene according to the desired structured light pattern. 
     
     
         8 . The sensor system of  claim 1 , further comprising an imaging sensor, wherein the imaging sensor comprises a plurality of photosensitive elements, wherein the plurality of photosensitive elements comprises at least one million photosensitive elements, wherein the operations further comprise causing the imaging sensor to provide information indicative of an image of the scene based on the structured light pattern. 
     
     
         9 . The sensor system of  claim 8 , wherein the operations further comprise determining a high-resolution depth map of the scene based on the depth map of the scene and the image of the scene. 
     
     
         10 . The sensor system of  claim 8 , wherein the at least one ToF sensor, the imaging sensor, and the at least one light source are coupled to a common substrate. 
     
     
         11 . The sensor system of  claim 1 , wherein the operations further comprise determining at least one inference about the scene based on the depth map of the scene. 
     
     
         12 . The sensor system of  claim 11 , wherein the at least one inference comprises information about objects in an environment of a vehicle or an operating context of the vehicle. 
     
     
         13 . The sensor system of  claim 11 , wherein the controller comprises at least one deep neural network, wherein the determining the at least one inference is performed by the at least one deep neural network. 
     
     
         14 . A system comprising:
 a plurality of sensor systems configured to be coupled to a vehicle, wherein each sensor system comprises:   at least one time-of-flight (ToF) sensor;   at least one imaging sensor, wherein the at least one ToF sensor and the at least one imaging sensor are configured to receive light from a scene;   at least one light source configured to emit a structured light pattern; and   a controller that carries out operations, the operations comprising:   dynamically adjusting the structured light pattern based on one or more retroreflective regions in the scene;   causing the at least one light source to illuminate at least a portion of the scene with the structured light pattern;   causing the at least one ToF sensor to provide time of flight information indicative of a depth map of the scene based on the structured light pattern; and   causing the imaging sensor to provide information indicative of an image of the scene based on the structured light pattern.   
     
     
         15 . The system of  claim 14 , wherein the operations further comprise determining a high-resolution depth map of the scene based on the depth map of the scene and the image of the scene. 
     
     
         16 . The system of  claim 14 , wherein at least one of the sensor systems comprises at least one ToF sensor and at least one imaging sensor in a common housing. 
     
     
         17 . A method comprising:
 dynamically adjusting a structured light pattern based on one or more retroreflective regions in a scene;   causing at least one light source to illuminate the scene with the structured light pattern;   receiving, from a time-of-flight (ToF) sensor, time of flight information about the scene based on the structured light pattern;   determining a depth map of the scene based on the received information; and   determining at least one inference about the scene based on the depth map of the scene.   
     
     
         18 . The method of  claim 17 , wherein the at least one inference comprises information about objects in an environment of a vehicle or an operating context of the vehicle. 
     
     
         19 . The method of  claim 17 , wherein dynamically adjusting the structured light pattern comprises selecting a desired structured light pattern from among a plurality of possible structured light patterns, wherein causing the at least one light source to illuminate the scene with the structured light pattern comprises illuminating the scene according to the desired structured light pattern. 
     
     
         20 . The method of  claim 17 , wherein dynamically adjusting the structured light pattern further comprises adjusting the structured light pattern based on an amount of ambient light or a time of day.