Image capture using three-dimensional reconstruction

Abstract

Embodiments may take the form of three-dimensional image sensing devices configured to capture an image including one or more objects. In one embodiments, the three-dimensional image sensing device includes a first image device configured to capture a first image and extract depth information for the one or more objects. Additionally, the image sensing device includes a second imaging device configured to capture a second image and determine an orientation of a surface of the one or more objects.

1 . A three-dimensional imaging apparatus configured to capture at least one image including one or more objects, comprising:
 a first sensor for capturing a polarized image, the first sensor including a first imaging device and a polarized filter associated with the first imaging device;   a second sensor for capturing a first non-polarized image;   a third sensor for capturing a second non-polarized image; and   at least one processing module for deriving depth information for the one or more objects utilizing at least the first non-polarized image and the second non-polarized image, the processing module further operative to combine the polarized image, the first non-polarized image, and the second non-polarized image to form a composite three-dimensional image.   
     
     
         2 . The three-dimensional imaging apparatus of  claim 1 , wherein the first sensor is positioned between the second and third sensors such that a blind region of the first sensor is between blind regions of the second and third sensors. 
     
     
         3 . The three-dimensional imaging apparatus of  claim 1 , wherein:
 the first sensor is a luminance sensor;   the second sensor is a first chrominance sensor; and   the third sensor is a second chrominance sensor.   
     
     
         4 . The three-dimensional imaging apparatus of  claim 3 , wherein a field of view of the first sensor is offset from both a field of view of the second sensor and a field of view of the third sensor. 
     
     
         5 . The three-dimensional imaging apparatus of  claim 3 , wherein:
 the polarized image is a polarized luminance image;   the first non-polarized image is a first chrominance image;   the second non-polarized image is a second chrominance image;   the at least one processing module is configured to generate a stereo disparity map from at least the first and second chrominance images; and   the at least one processing module is configured to derive depth information at least partially from the stereo disparity map.   
     
     
         6 . The three-dimensional imaging apparatus of  claim 5 , further comprising a fourth sensor configured to capture a second luminance image; wherein
 the at least one processing module is further configured to refine the stereo disparity map based on the second luminance image.   
     
     
         7 . The three-dimensional imaging apparatus of  claim 1 , wherein the polarized filter comprises an array of polarizing subfilters. 
     
     
         8 . The three-dimensional imaging apparatus of  claim 7 , wherein:
 the first sensor comprises at least one pixel; and   a first polarized subfilter of the array of polarizing subfilters overlays the at least one pixel.   
     
     
         9 . The three-dimensional imaging apparatus of  claim 8 , wherein the first polarized subfilter of the array of polarizing subfilters has a different type of polarization than a second polarized subfilter of the array of polarizing subfilters. 
     
     
         10 . The three-dimensional imaging apparatus of  claim 8 , wherein:
 the at least one pixel receives polarized light reflected from an imaged object, the polarized light corresponding to a polarization type of the first polarized subfilter; and   the first sensor determines a surface normal of the imaged object by measuring a polarization of the light received by the at least one pixel.   
     
     
         11 . The three-dimensional imaging apparatus of  claim 10 , further comprising:
 at least a second pixel adjacent to the at least one pixel; wherein   the second polarized subfilter overlays the at least a second pixel; and   the first sensor determines a surface normal of the imaged object by measuring a polarization of the light received by the at least a second pixel and comparing it to the polarization of the light received by the at least one pixel.   
     
     
         12 . The three-dimensional imaging apparatus of  claim 8 , wherein the luminance imaging device includes at least one additional pixel that corresponds to an unpolarized area of the polarized filter. 
     
     
         13 . The three-dimensional imaging apparatus of  claim 12 , wherein the polarized luminance image is a high dynamic range image created from a first luminance image recorded at least by the at least one pixel and a second luminance image recorded at least by the at least one additional pixel. 
     
     
         14 . The three-dimensional imaging apparatus of  claim 8 , wherein:
 the first sensor includes a microlens array; and   at least one microlens of the microlens array corresponds to the at least one pixel and is configured to focus light onto the at least one pixel.   
     
     
         15 . The three-dimensional imaging apparatus of  claim 1 , wherein the at least one processing module is further configured to identify at least one face in the composite three-dimensional image utilizing at least one of the surface information or the depth information. 
     
     
         16 . A three-dimensional imaging apparatus configured to capture at least one image including one or more objects, comprising:
 a first sensor for capturing a polarized chrominance image and determining surface information for the one or more objects, the first sensor including a color imaging device and a polarized filter associated with the color imaging device;   a second sensor for capturing a first luminance image;   a third sensor for capturing a second luminance image; and   at least one processing module for deriving depth information for the one or more objects utilizing at least the first luminance image and the second luminance image and combining the polarized chrominance image, the first luminance image, and the second luminance image to form a composite three-dimensional image utilizing the surface information and the depth information.   
     
     
         17 . A method for capturing at least one image of an object, comprising:
 capturing a polarized image of the object;   capturing a first non-polarized image of the object;   capturing a second non-polarized image of the object;   deriving depth information for the object from at least the first non-polarized image and the second non-polarized image;   determining a plurality of surface normals for the object, the plurality of surface normals derived from the polarized image;   creating a three-dimensional image from the depth information and the plurality of surface normals.   
     
     
         18 . The method of  claim 17 , wherein the operation of deriving depth information for the object comprises creating a stereo disparity from the first non-polarized image and the second non-polarized image. 
     
     
         19 . The method of  claim 17 , further comprising:
 determining, based on the surface normals, a simulated lighting of the object; and   altering the three-dimensional image to insert the simulated lighting of the object.   
     
     
         20 . The method of  claim 17 , wherein the operation of determining a plurality of surface normals for the object comprises:
 grouping each pixel of a pixel array into a subarray with at least one other pixel;   evaluating the polarized light received by the subarray; and   based on the evaluation, assigning a surface normal to a portion of the image recorded by the subarray.   
     
     
         21 . The method of  claim 17 , wherein the polarized image and first non-polarized image are captured by a single sensor.