System and method for frequency-based 3d reconstruction of objects

Abstract

Various embodiments are described herein for a system and method for performing 3D reconstruction of an object. In at least one example embodiment, the method may comprise obtaining image data of the object while projecting frequency-based light patterns on the object; locating candidates for correspondence points in the image data; selecting reflection points by applying a labeling method to the located candidates; and generating the 3D reconstruction of a surface of the object using the selected reflection points.

1 . A method of generating a 3D reconstruction of an object, wherein the method comprises:
 obtaining image data of the object while projecting frequency-based light patterns on the object;   locating candidates for correspondence points in the image data;   selecting reflection points by applying a labeling method to the located candidates; and   generating the 3D reconstruction of a surface of the object using the selected reflection points.   
     
     
         2 . The method as claimed in  claim 1 , wherein a light source is used to generate the frequency-based light patterns and an image capture device is used to obtain the image data, wherein the light source and the image capture device are on a common side of the object. 
     
     
         3 . The method as claimed in  claim 2 , wherein a position of the light source, the object and the image capture device are adjusted to acquire more reflected light from the object during image acquisition. 
     
     
         4 . The method as claimed in  claim 1 , wherein the object is moved further from a background to reduce noise in the obtained image data. 
     
     
         5 . The method as claimed in  claim 1 , wherein the method further comprises at least one of outputting the 3D reconstruction of the object and storing the 3D reconstruction of the object. 
     
     
         6 . The method as claimed in  claim 1 , wherein the frequency based light patterns comprise alternating light and dark regions. 
     
     
         7 . The method as claimed in  claim 1 , wherein intensities in the frequency-based light patterns are chosen to have a large range to reduce noise in the image data. 
     
     
         8 . The method as claimed in  claim 1 , wherein the image data comprises a sequence of N images when N frequency-based light patterns are generated. 
     
     
         9 . The method as claimed in  claim 1 , wherein the method further comprises determining a region of interest for the object in the image data before locating candidates for the correspondence points. 
     
     
         10 . The method as claimed in  claim 1 , wherein locating candidates for correspondence points in the image data comprises:
 performing a frequency transform on the sequences of images in the image data to generate frequency data for pixels in the image data that may correspond to a surface of the object;   locating pixels having frequencies in the frequency data that correspond to points on the surface of the object receiving similar frequencies when illuminated by the frequency-based light pattern; and   performing triangulation on the set of located pixels to locate the candidates for the correspondence points.   
     
     
         11 . The method as claimed in  claim 1 , wherein selecting the reflection points comprises:
 labeling all the candidates for the correspondence points;   defining an energy function based on at least one property for the candidate points; and   choosing the labelling that minimizes the total energy of the energy function.   
     
     
         12 . The method as claimed in  claim 11 , wherein the energy function is based on the Markov Random Field. 
     
     
         13 . The method as claimed in  claim 11 , wherein the energy function comprises:
 a data term that represents the distance from each candidate to a centre of the image capture device; and   a smoothness term that indicates the difference of the distances from the centre of the image capture device to the candidate and neighbouring points of the candidate.   
     
     
         14 . The method as claimed in  claim 10 , wherein the candidates for correspondence points are located for correspondences for the image capture device relative to the light source and from the light source relative to the image capture device. 
     
     
         15 . The method as claimed in  claim 14 , wherein when several located pixels in the image data correspond to a given point on the surface of the object, an average position of the located pixels is used as a correspondence to the given point. 
     
     
         16 . A computer readable medium comprising a plurality of instructions that are executable on a microprocessor of a device for adapting the device to implement a method of generating a 3D reconstruction of an object, wherein the method comprises:
 obtaining image data of the object while projecting frequency-based light patterns on the object;   locating candidates for correspondence points in the image data;   selecting reflection points by applying a labeling method to the located candidates; and   generating the 3D reconstruction of a surface of the object using the selected reflection points.   
     
     
         17 . The computer readable medium of  claim 16 , for locating candidates for correspondence points in the image data comprises, the method further comprises:
 performing a frequency transform on the sequences of images in the image data to generate frequency data for pixels in the image data that may correspond to a surface of the object;   locating pixels having frequencies in the frequency data that correspond to points on the surface of the object receiving similar frequencies when illuminated by the frequency-based light pattern; and   performing triangulation on the set of located pixels to locate the candidates for the correspondence points.   
     
     
         18 . The computer readable medium of  claim 16 , for selecting the reflection points, the method further comprises:
 labeling all the candidates for the correspondence points;   defining an energy function based on at least one property for the candidate points; and   choosing the labelling that minimizes the total energy of the energy function.   
     
     
         19 . The computer readable medium of  claim 18 , wherein the method comprises defining the energy function using a data term that represents the distance from each candidate to a centre of the image capture device; and a smoothness term that indicates the difference of the distances from the centre of the image capture device to the candidate and neighbouring points of the candidate. 
     
     
         20 . An electronic device for generating a 3D reconstruction of an object, the electrical device comprising:
 an input for receiving image data of the object that was obtained while frequency-based light patterns were projected on the object;   a processing unit coupled to the input, the processing unit being configured to locate candidates for correspondence points in the image data, select reflection points by applying a labeling method to the located candidates; and generate output data comprising the 3D reconstruction of a surface of the object using the selected reflection points; and   an output coupled to the processing unit to provide the output data.   
     
     
         21 . The electronic device of  claim 20 , wherein the processing unit is further configured to output the 3D reconstruction of the object and/or store the 3D reconstruction of the object. 
     
     
         22 . The electronic device of  claim 20 , wherein the processing unit is configured to control a light source to generate the frequency-based light patterns using a sequence of N images having alternating light and dark regions. 
     
     
         23 . The electronic device of  claim 20 , wherein the processing unit is configured to determine a region of interest for the object in the image data before locating candidates for the correspondence points. 
     
     
         24 . The electronic device of  claim 20 , wherein the processing unit is configured to locate candidates for correspondence points in the image data by:
 performing a frequency transform on the sequences of images in the image data to generate frequency data for pixels in the image data that may correspond to a surface of the object;   locating pixels having frequencies in the frequency data that correspond to points on the surface of the object receiving similar frequencies when illuminated by the frequency-based light pattern; and   performing triangulation on the set of located pixels to locate the candidates for the correspondence points.   
     
     
         25 . The electronic device of  claim 20 , wherein the processing unit is configured to select the reflection points by:
 labeling all the candidates for the correspondence points;   defining an energy function based on at least one property for the candidate points; and   choosing the labelling that minimizes the total energy of the energy function.   
     
     
         26 . The electronic device of  claim 25 , wherein the energy function comprises:
 a data term that represents the distance from each candidate to a centre of the image capture device; and   a smoothness term that indicates the difference of the distances from the centre of the image capture device to the candidate and neighbouring points of the candidate.   
     
     
         27 . A system for generating a 3D reconstruction of an object, wherein the system comprises:
 a light source for generating frequency-based light patterns;   an image capture device for obtaining image data when the frequency based light patterns are projected to the object; and   an electronic device that controls the light source and the image capture device and comprises an image analysis module that is configured to locate candidates for correspondence points in the image data, select reflection points by applying a labeling method to the located candidates; and generate output data comprising the 3D reconstruction of a surface of the object using the selected reflection points.   
     
     
         28 . The system of  claim 27 , wherein the image analysis module is further configured to output the 3D reconstruction of the object and/or store the 3D reconstruction of the object. 
     
     
         29 . The system of  claim 27 , wherein the image analysis module is configured to locate candidates for correspondence points in the image data by performing a frequency transform on the sequences of images in the image data to generate frequency data for pixels in the image data that may correspond to a surface of the object; locating pixels having frequencies in the frequency data that correspond to points on the surface of the object receiving similar frequencies when illuminated by the frequency-based light pattern; and performing triangulation on the set of located pixels to locate the candidates for the correspondence points. 
     
     
         30 . The system of  claim 27 , wherein the image analysis device is configured to select the reflection points by: labeling all the candidates for the correspondence points; defining an energy function based on at least one property for the candidate points; and choosing the labelling that minimizes the total energy of the energy function.