Method and system for determining optimal exposure of structured light based 3D camera

Abstract

Provided are a method and system for determining an optimal exposure of a structured light based 3D camera. The system includes a projecting means for illuminating a predetermined pattern on a target object, an image capturing means for capturing an image of the target object with the pattern projected, and a processing means for reconstructing 3D data for the target object by identifying the pattern of the captured images from the image capturing means. The system automatically determines an optimal exposure of the structured light based 3D camera system through analyzing the captured image of the target object.

What is claimed is: 
     
       1. A method for determining an optimal exposure of a structured light based three dimensional (3D) camera system including a projecting means for illuminating a predetermined pattern on a target object, an image capturing means for capturing an image of the target object with the pattern projected thereon, and a processing means for reconstructing 3D data based on the captured image, wherein the method comprises the steps of:
 obtaining an image of a target object with a predetermined pattern projected thereon, which is illuminated from a projecting means, and an image of the target object without a predetermined pattern projected thereon; and 
 determining automatically an optimal exposure level of the structured light based 3D camera system using said two kinds of images, said image of the target object with the predetermined pattern projected thereon and said image of the target object without the predetermined pattern projected thereon; and 
 controlling an exposure level of the structured light based 3D camera system based on the determined optimal exposure level, 
 wherein the determining automatically the optimal exposure level of the structured light based 3D camera system using said image of the target object with the predetermined pattern projected thereon and said image of the target object without the predetermined pattern projected thereon includes processing data from at least one characteristic curve associated with the image of the target object with the predetermined pattern projected thereon and processing data from at least one characteristic curve associated with the image of the target object without the predetermined pattern projected thereon to determine the optimal exposure level. 
 
     
     
       2. The method of  claim 1 , wherein the step of determining automatically an optimal exposure level comprises
 a) dividing an intensity of a corresponding pixel into red intensity, green intensity, and blue intensity of three channels (R, G, B) for one same pixel at each of the captured images, and generating a characteristic curve for each of the three channels for showing the intensity of a pixel for each channel according to variation of exposure levels; 
 b) estimating intensity I of the corresponding pixel according to variation of exposure levels using the generated characteristic curves, and calculating an intensity difference ΔI or a brightness ratio SNR between the image of the target object with the predetermined pattern projected and the image of the target object without the predetermined pattern for the corresponding pixel using the estimated intensity; 
 c) collecting brightness data about intensity differences ΔI or brightness ratios SNR according to variation of exposure levels for all pixels by repeating the steps a) and b) for all pixels of each image; 
 d) discriminating pixels with spread phenomenon (saturation) occurred or the pixels without the predetermined pattern projected when the projecting means projects the predetermined pattern from the collected brightness data and excluding the brightness data of the discriminated pixels; and 
 e) calculating the number of pixels having an intensity difference ΔI and a brightness ratio SNR exceeding a predetermined threshold ΔI th  or SNR th  at every exposure levels using the brightness data filtered at the step d) and deciding an exposure level having a largest calculated number of pixels as an optimal exposure level. 
 
     
     
       3. The method of  claim 2 , wherein the characteristic curves are created based on proportional relation between an intensity and an exposure, where the intensity denotes brightness of a pixel and the exposure is an amount of incoming light. 
     
     
       4. The method of  claim 3 , wherein in the step a), proportional factors k R , k G , and k B  are calculated by applying an exposure applied for capturing the images in the image obtaining step and red intensity, green intensity, and blue intensity for one same pixel in each of the obtained images to an equation: I R =k R *E, I G =k G *E, I B =k B *E, intensities of a corresponding pixel are calculated for input exposures by inputting various exposures to the equation, and the characteristic curve is created by corresponding the calculated intensities of a corresponding pixel with the input exposures, where I R , and I G  and I B  denote the intensities in R, G, and B channels, k R , k G , and k B  denote a proportional factor, and E denotes an exposure in the equation. 
     
     
       5. The method of  claim 4 , wherein in the step b), intensity of a gray level of a corresponding pixel is estimated according to variation of exposure levels by transforming the created characteristic curves for red, green, and blue channels to a characteristic curve of a gray level. 
     
     
       6. The method of  claim 5 , wherein in the step d), an
 enable term is set as a factor for checking the collected brightness data, is checked as Disable for a pixel in which spread phenomenon (saturation) occurs when the projecting means illuminates the predetermined pattern and an intensity difference ΔI or a brightness ratio SNR of the pixel is classified as unreliable data. 
 
     
     
       7. The method of  claim 6 , wherein in the step e), the brightness threshold ΔI th  or SNR th  for the intensity difference or the brightness ratio is set as minimum intensity difference or minimum brightness ratio for discriminating an area with the predetermined pattern illuminated from an area without the predetermined pattern illuminated. 
     
     
       8. The method of claim 1, further comprising adjusting an exposure level of the image capturing means by adjusting at least one of an aperture of an iris and a shutter speed. 
     
     
       9. The method of claim 1, wherein the data from the at least one characteristic curve associated with the image of the target object with the predetermined pattern projected thereon and the data from the at least one characteristic curve associated with the image of the target object without the predetermined pattern projected thereon includes at least one of pixel intensity data and exposure level data. 
     
     
       10. The method of claim 1, wherein the processing data from the at least one characteristic curve associated with the image of the target object with the predetermined pattern projected thereon includes identifying pixels of the image having an intensity difference exceeding a threshold difference. 
     
     
       11. The method of claim 1, wherein the processing data from the at least one characteristic curve associated with the image of the target object without the predetermined pattern projected thereon includes identifying pixels of the image having an intensity difference exceeding a threshold difference. 
     
     
       12. The method of claim 1, wherein the processing data from the at least one characteristic curve associated with the image of the target object with the predetermined pattern projected thereon includes identifying pixels of the image having a signal to brightness ratio exceeding a threshold brightness ratio. 
     
     
       13. The method of claim 1, wherein the processing data from the at least one characteristic curve associated with the image of the target object without the predetermined pattern projected thereon includes identifying pixels of the image having a brightness ratio exceeding a threshold brightness ratio. 
     
     
       14. The method of claim 1, wherein the projecting means comprises a beam projector. 
     
     
       15. The method of claim 1, wherein the image capturing means comprises a camera. 
     
     
       16. The method of claim 1, wherein the processing means comprises a computer. 
     
     
       17. A method comprising:
 projecting, via a projecting means of a structured light based 3D camera system, a predetermined pattern on a target object;   obtaining, via an image capturing means of the structured light based three dimensional (3D) camera system, a first image of the target object without the predetermined pattern projected thereon, the first image including a first pixel and a second pixel;   obtaining, via the image capturing means, a second image of the target object with the predetermined pattern projected thereon, the second image including a third pixel corresponding to the first pixel of the first image and a fourth pixel corresponding to the second pixel of the first image, the structured light based 3D camera system including a processing means for reconstructing 3D data based on the first and second images;   determining automatically an optimal exposure level of the structured light based 3D camera system using the first image and the second image; and   controlling an exposure level of the structured light based 3D camera system based on the determined optimal exposure level,   wherein the determining automatically the optimal exposure level of the structured light based 3D camera system using the first image and the second image includes
 determining, for each of a plurality of exposure levels, a difference between an intensity of the first pixel and an intensity of the second pixel, 
 determining, for each of the plurality of exposure levels, a difference between an intensity of the third pixel and an intensity of the fourth pixel, and 
 determining one of the plurality of exposure levels as the optimal exposure level based on the determined differences. 
   
     
     
       18. The method of claim 17, wherein the processing means comprises a computer. 
     
     
       19. The method of claim 17, further comprising adjusting an exposure level of the image capturing means by adjusting at least one of an aperture of an iris and a shutter speed. 
     
     
       20. The method of claim 17, wherein the projecting means comprises a beam projector. 
     
     
       21. The method of claim 17, wherein the image capturing means comprises a camera. 
     
     
       22. A method comprising:
 projecting, via a projecting means of a structured light based 3D camera system, a predetermined pattern on a target object;   obtaining, via an image capturing means of the structured light based three dimensional (3D) camera system, a first image of the target object without the predetermined pattern projected thereon, the first image including a first pixel and a second pixel;   obtaining, via the image capturing means, a second image of the target object with the predetermined pattern projected thereon, the second image including a third pixel corresponding to the first pixel of the first image and a fourth pixel corresponding to the second pixel of the first image, the structured light based 3D camera system including a processing means for reconstructing 3D data based on the first and second images;   determining automatically an optimal exposure level of the structured light based 3D camera system using the first image and the second image; and   controlling an exposure level of the structured light based 3D camera system based on the determined optimal exposure level,   wherein the determining automatically the optimal exposure level of the structured light base 3D camera system using the first image and the second image includes
 determining, from the first image and the second image, estimated values of pixel intensities for a plurality of exposure levels, and 
 determining one of the plurality of exposure levels as the optimal exposure level based on the estimated values of pixel intensities. 
   
     
     
       23. The method of claim 22, wherein the image capturing means comprises a camera. 
     
     
       24. The method of claim 22, wherein the processing means comprises a computer. 
     
     
       25. The method of claim 22, further comprising adjusting an exposure level of the image capturing means by adjusting at least one of an aperture of an iris and a shutter speed. 
     
     
       26. The method of claim 22, wherein the projecting means comprises a beam projector.