US11029144B2ActiveUtilityPatentIndex 50
Super-rapid three-dimensional topography measurement method and system based on improved fourier transform contour technique
Est. expiryMar 24, 2037(~10.7 yrs left)· nominal 20-yr term from priority
G06T 7/521G01B 11/2536G01B 11/2504G01B 11/254
50
PatentIndex Score
0
Cited by
17
References
10
Claims
Abstract
A super-rapid three-dimensional measurement method and system based on an improved Fourier transform contour technique is disclosed. The method comprises: firstly calibrating a measurement system to obtain calibration parameters, then cyclically projecting 2n patterns into a measured scene using a projector, wherein n patterns are binary sinusoidal fringes with different high frequency, and the other n patterns are all-white images with the values of 1, and projecting the all-white images between every two binary high-frequency sinusoidal fringes, and synchronously acquiring images using a camera; and then performing phase unwrapping on wrapped phases to obtain initial absolute phases, and correcting the initial absolute phases, and finally reconstructing a three-dimensional topography of the measured scene by exploiting the corrected absolute phases and the calibration parameters to obtain 3D spatial coordinates of the measured scene in a world coordinate system, thereby accomplishing three-dimensional topography measurement of an object. In this way, the precision of three-dimensional topography measurement is ensured, and the speed of three-dimensional topography measurement is improved.
Claims
exact text as granted — not AI-modifiedThe invention claimed is:
1. A three-dimensional topography measurement method based on an improved Fourier transform contour technique, the method comprising the steps of:
firstly calibrating a measurement system to obtain calibration parameters, the measurement system being composed of a projector, a camera and a computer;
cyclically projecting n (n≥2) patterns and m (m=n) all-white images into a measured scene using the projector,
wherein the n patterns are binary sinusoidal fringes with different high frequency fringes, and the m all-white images are with values of 1;
projecting the m all-white images between every two binary high-frequency sinusoidal fringes and synchronously acquiring n+m images using the camera;
using a background normalized Fourier transform contour method to obtain a wrapped phase;
using temporal phase unwrapping with projection distance minimization (PDM) method to obtain initial absolute phases;
using a reliability guided compensation (RGC) of fringe order error method to correct the initial absolute phase; and
finally reconstructing a three-dimensional topography of the measured scene with the corrected absolute phases and the calibration parameters to obtain 3D spatial coordinates of the measured scene in a world coordinate system, thereby accomplishing three-dimensional topography measurement of an object.
2. The method according to claim 1 , wherein the step of projecting and synchronously acquiring comprises:
emitting n high-frequency sinusoidal fringe by the projector, said n high-frequency sinusoidal fringe being different in wavelength and the wavelength being marked as {λ 1 , λ 2 , . . . , λ n };
designing the wavelength to meet two conditions including a first condition that the wavelength of the sinusoidal fringe is small enough to ensure the phase can be successfully retrieved by traditional Fourier transform contour technique; and a second condition that a least common multiple of the wavelength is greater than or equal to a resolution of the projector along a sinusoidal intensity value, denoted as W, wherein the following formula is satisfied:
LCM(λ 1 ,λ 2 , . . . ,λ n )≥ W (1),
where LCM represents the least common multiple operation and generated high-frequency sinusoidal fringes are represented by the following formula in the projector space:
I p ( x p ,y p )= a p +b p cos(2π f 0 p x p ) (2),
where superscript p represents the projector space, and I p represents an intensity of fringes, (x p ,y p ) is pixel coordinates of the projector, a p is an average intensity of the sinusoidal fringes, b p is an amplitude of the sinusoidal fringes and f 0 p is a frequency of the sinusoidal fringes;
using the halftone technique to convert the high-frequency sinusoidal fringes into binary high-frequency sinusoidal fringes, so that a projection speed of the projector can reach a maximum of an inherent projection speed of the projector, ensuring that hardware does not affect a measurement speed,
wherein when the fringe pattern is a binary pattern, both a p and b p in the equation (2) are ½, and the equation (2) is written as:
I 1 p ( x p ,y p )=½+½ cos(2π f 0 p x p ) (3),
where I 1 p represents an intensity of a first high-frequency sinusoidal fringe pattern, the all-white images projected between every two binary high-frequency sinusoidal fringes mean that the values of all the pixels on the projected image are “1”, that is, all micro-mirrors on the digital micro-mirror device (DMD) as a key component of digital light processing (DLP) projection system, are an “on” state and are represented by the following formula:
I 2 p ( x p ,y p )=1 (4),
where I 2 p represents an intensity of the all-white images, (x p ,y p ) represents the pixel coordinates of the projector,
wherein an expression of the remaining high-frequency sinusoidal fringe is the same as the formula (3), except that the frequency f 0 p is different according to the different wavelength, and
wherein the n+m images are cyclically projected into the measured scene using the projector, and the camera synchronously acquires the n+m image using a trigger signal of the projector.
3. The method according to claim 1 , wherein the wrapped phase is obtained by using a background normalized Fourier transform contour technique method, and a process of obtaining the wrapped phase comprises the steps of:
after the acquisition of the n+m images by the camera, every two images are sequentially processed, wherein the every two images includes a high-frequency sinusoidal fringe image and a corresponding all-white image and the high-frequency sinusoidal fringe image and the corresponding all-white image captured by the camera are respectively expressed by the following formulas:
I 1 ( x c ,y c )=½α( x c ,y c )+½α( x c ,y c )cos[2π f 0 x c +ϕ( x c ,y c )] (5), and
I 2 ( x c ,y c )=α( x c ,y c ) (6),
where superscript c represents a camera space, I 1 is an image captured by the camera after the high-frequency sinusoidal fringe pattern is projected onto the measured scene, I 2 is an image captured by the camera after the all-white image is projected onto the measured scene, (x c ,y c ) is pixel coordinates of the camera, α(x c ,y c ) is a surface reflectivity of the measured object, f 0 is a sinusoidal fringe frequency, ϕ(x c ,y c ) is the phase containing a depth information of the object, ½α(x c ,y c ) is a zero-frequency part after performing Fourier transform, wherein by using I 1 and I 2 , an influence of the zero-frequency part and the surface reflectivity α(x c ,y c ) of the object to be measured can be removed before performing the Fourier transform, according to the following equation (7):
I
d
(
x
c
,
y
c
)
=
2
I
1
-
l
2
I
2
+
γ
=
cos
[
2
π
f
0
x
c
+
ϕ
(
x
c
,
y
c
)
]
,
(
7
)
where γ is a constant;
performing the Fourier transform on the I d after background normalization;
using a filter to extract a valid information; and
obtaining the wrapped phase by performing an inverse Fourier transform, such that wrapped phases corresponding to each high-frequency sinusoidal fringe acquired by the camera are obtained and contain the depth information of the scene corresponding to each moment when the camera captures the high-frequency sinusoidal fringe pattern.
4. The method according to claim 1 , wherein the initial absolute phase is obtained by using a temporal phase unwrapping with projection distance minimization (PDM) method, and a process of obtaining the initial absolute phase comprises the steps of:
using the wrapped phases corresponding to a set of high-frequency sinusoidal fringes to unwrap each of wrapped phases, wherein the high-frequency sinusoidal fringes projected by the projector are different in wavelength, and are recorded as a wavelength vector λ=[λ 1 , λ 2 , . . . , λ n ] T , wrapped phase vector corresponding to each high-frequency sinusoidal fringe obtained by Fourier transform contour technique method is marked as φ=[ϕ 1 , ϕ 2 , . . . , ϕ n ] T , fringe order combinations are listed one by one, each set of fringe-level sub-vectors is recorded as k i , which contains a corresponding fringe order of each wrapped phase [k 1 , k 2 , . . . , k n ] T , and for each fringe order vector k i , a corresponding absolute phase Φ i is calculated by the following formula:
Φ i =φ+2π k i (8),
where Φ i is an absolute phase vector, φ is the wrapped phase vector, k i is the fringe order sub-vector;
calculating a projection point vector of the absolute phase by equations (9) and (10):
t
=
(
λ
-
1
2
)
1
(
λ
-
1
)
T
Φ
i
=
(
∑
j
=
1
n
(
1
λ
j
2
)
)
-
1
∑
j
=
1
n
Φ
j
λ
i
,
and
(
9
)
P
i
=
t
λ
i
-
1
,
(
10
)
where λ i is the wavelength vector, Φ i is the absolute phase vector, n is number of projected sinusoidal fringes, P i is the projection point vector;
obtaining a distance d i 2 between Φ i and P i by the formula (11):
d i 2 =∥P i −Φ i ∥ 2 =( P i −Φ i ) T ( P i −Φ i ) (11);
selecting a fringe-order sub-vector corresponding to a minimum distance d min 2 as an optimal solution; and
obtaining the absolute phase Φ corresponding to the optimal solution as the initial absolute phase.
5. The method according to claim 4 , wherein a range of enumerated fringe-level sub-combinations is further reduced by depth constraint, comprising the steps of:
firstly estimating a depth range of a measured scene [z min w ,z max w ], wherein z min w is a minimum value of a depth of a measurement range in the world coordinate system, z max w is a maximum value of the depth of the measurement range in the world coordinate system;
obtaining a range of a phase distribution according to the calibration parameters and a depth constraint method [Φ min ,Φ max ], wherein Φ min is a minimum value of the absolute phase and Φ max is a maximum value of the absolute phase; and
obtaining a range of fringe order by the following formula:
k
min
(
x
c
,
y
c
)
=
floor
[
Φ
min
(
x
c
,
y
c
)
2
π
]
,
and
(
12
)
k
max
(
x
c
,
y
c
)
=
ceil
[
Φ
max
(
x
c
,
y
c
)
2
π
]
,
(
13
)
where k min represents a minimum value of fringe order, k max represents a maximum value of fringe order, (x c ,y c ) represents pixel coordinates of the camera, floor represents a round-down operation, Φ min represents minimum value of the phase, ceil represents a rounding up operation, and Φ max represents maximum value of the phase.
6. The method according to claim 1 , wherein the initial absolute phase is corrected by using a reliability guided compensation (RGC) of fringe order error method, and a process of correcting the initial absolute phase comprises the steps of:
using a minimum projection distance corresponding to each pixel d min 2 as a basis for evaluating reliability of the absolute phase;
defining the reliability at a pixel boundary by a sum of the reliability of adjacent two pixels; and
determining a path to be processed by comparing the reliability value at the pixel boundary,
wherein a correction is performed from a pixel with a larger reliability value after the step of comparing, and the reliability value at an intersection of all pixels is stored in a queue, and is sorted according to an amount of credibility value, and
wherein the greater the credibility value is, the first it is processed, thus resulting in a corrected absolute phase.
7. The method according to claim 6 , wherein the RGC of fringe order error method comprises the steps of:
(1) calculating the credibility value of each pixel boundary, wherein the minimum projection distance d min 2 obtained by temporal phase unwrapping with projection distance minimization (PDM) method corresponding to the adjacent two pixels connected at the boundary are added as the credibility value at the pixel boundary;
(2) sequentially determining adjacent pixels, wherein when a absolute difference of the phase value corresponding to the adjacent pixels is less than π, the two pixels are grouped into one group, and all the pixels are grouped according to this method;
(3) sequentially correcting absolute phases according to the order of credibility values at the pixel boundaries, wherein the higher the credibility is, the first it is processed,
wherein when two connected pixels belong to the same group, no processing is performed,
wherein when two connected pixels belong to different groups and a number of pixels of the group with a small number of pixels is less than a threshold T h , all phase values in a smaller group are corrected according to a group with a larger number of pixels and two groups are combined, wherein phase values corresponding to the pixels belonging to the groups having a larger number of pixels and the smaller number of pixels are respectively Φ L and Φ S , and a value of
Round
(
Φ
L
-
Φ
S
2
π
)
multiplied by 2π is added to the phase value Φ S corresponding to all the pixels in the group having a smaller number of pixels, and the two groups are combined where Round means rounding off; and
(4) repeating the step (3) until all pixel boundaries in the queue have been processed.
8. The method according to claim 1 , wherein the step of reconstructing the three-dimensional topography comprises the step of:
combining the following formula with the calibration parameters and the corrected absolute phase Φ, and obtaining final three-dimensional world coordinates so as to complete the reconstruction:
x
p
=
Φ
W
2
π
N
L
Z
p
=
M
Z
+
N
Z
C
Z
x
p
+
1
X
p
=
E
X
Z
p
+
F
X
Y
p
=
E
Y
Z
p
+
F
Y
,
(
14
)
where E X , F X , E Y , F Y , M Z , N Z , C Z are intermediate variables, and Φ is the absolute phase, W is a resolution of the projector along a direction of fringe intensity variation, N L is a corresponding number of fringes, x p , is a projector coordinates, and X p , Y p , Z p are three-dimensional spatial coordinates of the measured object in the world coordinate system;
obtaining three-dimensional data of the measured scene at a current moment; and
repeatedly processing collected two-dimensional pattern sequences so as to obtain a three-dimensional topography reconstruction result of the scene for a whole measurement period.
9. A super-rapid three-dimensional topography measurement system based on an improved Fourier transform contour technique, the system comprising:
a measuring subsystem, the measuring system comprising a projector, a camera and a computer;
a Fourier transform contour technique subsystem; and
the computer being programmed to function as a calibration unit, a projection and acquisition image unit, and a three-dimensional reconstruction unit,
wherein:
the Fourier transform contour technique subsystem consists of a background normalized Fourier transform contour technique module, a temporal phase unwrapping technique with a projection distance minimization (PDM) module and a reliability guided compensation (RGC) of fringe order error module,
the calibration unit is configured to calibrate the measurement subsystem so as to obtain calibration parameters,
in the projection and acquisition image unit, the projector projects n patterns and m (m=n) all-white images cyclically to a measured scene, n≥2, wherein then patterns are binary high-frequency sinusoidal fringes with different wavelengths, and the m all-white images are with pixel value of 1, and projected between every two binary high-frequency sinusoidal fringes, and the n+m images are collected synchronously by the camera,
the background normalized Fourier transform contour technique module is configured to process the collected n+m images so as to get wrapped phases, the temporal phase unwrapping with the PDM module is configured to obtain a preliminary absolute phase, and the RGC of fringe order error module is configured to correct an initial absolute phase, and
the three-dimensional reconstruction unit is configured to reconstruct a three-dimensional topography of the measured scene with a corrected absolute phase and the calibration parameters to obtain three-dimensional spacial coordinates of the measured scene in a world coordinate system, thereby accomplishing three-dimensional topography measurement of an object.
10. The system according to claim 9 , wherein the background normalized Fourier transform contour technique module, after acquiring images collected by the camera, is further configured to sequentially process every two images including a high-frequency sinusoidal fringe and a corresponding all-white image wherein the high-frequency sinusoidal fringe image and the all-white image captured by the camera are respectively expressed by the following formulas:
I 1 ( x c ,y c )=½α( x c ,y c )+½α( x c ,y c )cos[2π f 0 x c +ϕ( x c ,y c )] (5), and
I 2 ( x c ,y c )=α( x c ,y c ) (6),
where superscript c represents a camera space, I I is an image captured by the camera after the high-frequency sinusoidal fringe pattern is projected onto the measured scene, I 2 is an image captured by the camera after the all-white image is projected onto the measured scene, (x c ,y c ) is pixel coordinates of the camera, α(x c ,y c ) is a surface reflectivity of the measured object, f 0 is the sinusoidal fringe frequency, ϕ(x c ,y c ) is the phase containing a depth information of the object, ½α(x c ,y c ) is a zero-frequency part after performing Fourier transform, wherein by using I 1 and I 2 , an influence of the zero-frequency part and the surface reflectivity α(x c ,y c ) of the object to be measured can be removed before performing the Fourier transform, according to the following equation (7):
I
d
(
x
c
,
y
c
)
=
2
I
1
-
l
2
I
2
+
γ
=
cos
[
2
π
f
0
x
c
+
ϕ
(
x
c
,
y
c
)
]
,
(
7
)
where γ is a constant,
wherein the background normalized Fourier transform contour technique module is further configured to carry out Fourier transform on the I d after background normalization,
wherein the system further comprises a filter used to extract a valid information, and is further configured to use an inverse Fourier transform to obtain the wrapped phase such that the wrapped phase corresponding to each high-frequency sinusoidal fringe acquired by the camera is obtained and contains the depth information of the scene corresponding to each moment when the camera captures the high-frequency sinusoidal fringe pattern;
wherein the temporal phase unwrapping technique with the projection distance minimization (PDM) module is further configured to use the wrapped phases corresponding to a set of sinusoidal fringes to unwrap each of wrapped phases, wherein the high-frequency sinusoidal fringes projected by the projector are different in wavelength, and are recorded as a wavelength vector λ=[λ 1 , λ 2 , . . . , λ n ] T , a wrapped phase vector corresponding to each high-frequency sinusoidal fringe obtained by Fourier transform contour technique is marked as φ=[ϕ 1 , ϕ 2 , . . . , ϕ n ] T , fringe order sub-combinations are listed one by one, each set of fringe order sub-vectors is recorded as k i which contains a corresponding fringe order of each wrapped phase [k 1 , k 2 , . . . , k n ] T , and for each fringe order vector k i , a corresponding absolute phase Φ i is calculated by the following formula:
Φ i =φ+2π k i (8),
where Φ i is an absolute phase vector, φ is the wrapped phase vector, k i is the fringe order sub-vector, and then a projection point vector of the absolute phase is calculated by equations (9) and (10):
t
=
(
λ
-
1
2
)
1
(
λ
-
1
)
T
Φ
i
=
(
∑
j
=
1
n
(
1
λ
j
2
)
)
-
1
∑
j
=
1
n
Φ
j
λ
i
,
and
(
9
)
P
i
=
t
λ
i
-
1
,
(
10
)
where λ i is the wavelength vector, Φ i is the absolute phase vector, n is number of projected sinusoidal fringes, P i is the projection point vector, and finally a distance d i 2 between Φ i and P i is obtained by the formula
d i 2 =∥P i −Φ i ∥ 2 =( P i −Φ i ) T ( P i −Φ i ) (11),
wherein the system is further configured to select a fringe-order sub-vector corresponding to a minimum distance d min 2 as an optimal solution, and then the absolute phase Φ corresponding to the optimal solution is obtained as the initial absolute phase, and
wherein the reliability guided compensation (RGC) of fringe order error module is further configured to:
use a minimum projection distance d min 2 corresponding to each pixel as a basis for evaluating reliability of absolute phase, wherein the reliability at a pixel boundary is defined by a sum of the credibility of adjacent two pixels; and
compare a reliability value at the pixel boundary, and determine a path to be processed, wherein a correction is performed from a pixel with a larger reliability value the reliability value at a intersection of all pixels is stored in a queue and sorted according to an amount of credibility value, and the greater the credibility value is, the first it is processed, thus resulting in a corrected absolute phase.Cited by (0)
No later patents cite this yet.
References (0)
No backward citations on record.