US2018020161A1PendingUtilityA1
Method and apparatus for obtaining panoramic and rectilinear images using rotationally symmetric wide-angle lens
Est. expiryJan 21, 2030(~3.5 yrs left)· nominal 20-yr term from priority
Inventors:Gyeongil Kweon
H04N 23/81H04N 23/698G06T 3/0018H04N 5/23238H04N 5/217G06T 3/047
49
PatentIndex Score
0
Cited by
0
References
0
Claims
Abstract
The present invention provides mathematically accurate image processing algorithms for extracting natural looking panoramic images and distortion-free rectilinear images from images acquired using a camera equipped with a wide-angle lens which is rotationally symmetric about an optical axis and devices implementing such algorithms. Imaging systems using this method can be used not only in security•surveillance applications for indoor and outdoor environments, but also in diverse areas such as video phone for apartment entrance door, rear view camera for vehicles, visual sensor for unmanned aerial vehicles and robots, camera phone, PC camera, and broadcasting camera. Also, it can be used to obtain panoramic or rectilinear photographs using a digital camera.
Claims
exact text as granted — not AI-modified1 . An imaging system comprising:
an image acquisition means for acquiring an uncorrected image plane using a wide-angle lens rotationally symmetric about an optical axis, an image processing means for extracting a panoramic image plane from the uncorrected image plane; and an image display means for displaying the panoramic image plane on a rectangular screen, wherein, if a coordinate of an image point on the rectangular screen corresponding to an object point having a coordinate (X, Y, Z) in a world coordinate system, which has a nodal point of the wide-angle lens as an origin and a vertical line passing through the origin as an Y-axis and an intersection line between a reference plane, which is a plane containing both the Y-axis and the optical axis of the wide-angle lens, and a horizontal plane perpendicular to the Y-axis as a Z-axis, is given as (x″, y″), the lateral coordinate x″ of the image point is given as x″=cψ, and the longitudinal coordinate y″ of the image point is given as y″=cF(δ), wherein, c is a proportionality constant, ψ is a horizontal incidence angle an incident ray originating from the object point makes with an Y-Z plane and is given as
ψ
=
tan
-
1
(
X
Z
)
,
δ is a vertical incidence angle the incident ray makes with a X-Z plane and is given as
δ
=
tan
-
1
(
Y
X
2
+
Z
2
)
,
and the function F(δ) is a monotonically increasing function of the vertical incidence angle.
2 . The imaging system of claim 1 ,
wherein the function F(δ) of the vertical incidence angle δ of the incident ray is given by any one among F(δ)=tan δ, F(δ)=δ and
F
(
δ
)
=
ln
{
tan
(
δ
2
+
π
4
)
}
.
3 . An imaging system comprising:
an image acquisition means for acquiring an uncorrected image plane using a fisheye lens rotationally symmetric about an optical axis, an image processing means for extracting a panoramic image plane from the uncorrected image plane; and an image display means for displaying the panoramic image plane on a rectangular screen, wherein, if a coordinate of an image point on the rectangular screen corresponding to an object point having a coordinate (X, Y, Z) in a world coordinate system, which has a nodal point of the fisheye lens as an origin and a vertical line passing through the origin as an Y-axis and an intersection line between a reference plane, which is a plane containing both the Y-axis and the optical axis of the fisheye lens, and a horizontal plane perpendicular to the Y-axis as a Z-axis, is given as (x″, y″), the lateral coordinate x″ of the image point is proportional to a horizontal incidence angle ψ, herein, the horizontal incidence angle iv is an angle an incident ray originating from the object point makes with an Y-Z plane and is given as
ψ
=
tan
-
1
(
X
Z
)
,
and straight lines parallel to the Y-axis are captured as straight lines parallel to an y″-axis on the screen.
4 . The imaging system of claim 3 further comprising an image selection means.
5 . The imaging system of claim 4 ,
wherein the image selection means comprise a situation awareness module.
6 . The imaging system of claim 4 ,
wherein the image selection means comprise a direction sensing means.
7 . An imaging system comprising:
an image acquisition means for acquiring an uncorrected image plane using a fisheye lens rotationally symmetric about an optical axis, an image processing means for extracting a panoramic image plane from the uncorrected image plane; and an image display means for displaying the panoramic image plane on a rectangular screen, wherein, if the optical axis of the fisheye lens is aligned parallel to a horizontal plane, and a coordinate of an image point on the rectangular screen corresponding to an object point having a coordinate (X, Y, Z) in a world coordinate system, which has a nodal point of the fisheye lens as an origin and a vertical line passing through the origin as an Y-axis and the optical axis of the fisheye lens as a Z-axis, is given as (x″, y″), the lateral coordinate x″ of the image point is proportional to a horizontal incidence angle ψ, herein, the horizontal incidence angle iv is an angle an incident ray originating from the object point makes with an Y-Z plane and is given as
ψ
=
tan
-
1
(
X
Z
)
,
and the longitudinal coordinate y″ of the image point is given as a monotonically increasing function F(δ) of a vertical incidence angle δ,
herein, the vertical incidence angle δ is an angle the incident ray makes with a X-Z plane and is given as
δ
=
tan
-
1
(
Y
X
2
+
Z
2
)
.
8 . The imaging system of claim 7 ,
wherein the function F(δ) of the vertical incidence angle δ of the incident ray is given by any one among F(δ)=tan δ, F(δ)=δ and
F
(
δ
)
=
ln
{
tan
(
δ
2
+
π
4
)
}
.
9 . An imaging system comprising:
an image acquisition means for acquiring an uncorrected image plane using a fisheye lens rotationally symmetric about an optical axis, an image processing means for extracting a panoramic image plane from the uncorrected image plane; and an image display means for displaying the panoramic image plane on a rectangular screen, wherein, if the optical axis of the fisheye lens is aligned parallel to a horizontal plane, and a coordinate of an image point on the rectangular screen corresponding to an object point having a coordinate (X, Y, Z) in a world coordinate system, which has a nodal point of the fisheye lens as an origin and a vertical line passing through the origin as an Y-axis and the optical axis of the fisheye lens as a Z-axis, is given as (x″, y″), the lateral coordinate x″ of the image point is proportional to a horizontal incidence angle ψ, herein, the horizontal incidence angle iv is an angle an incident ray originating from the object point makes with an Y-Z plane and is given as
ψ
=
tan
-
1
(
X
Z
)
,
and straight lines parallel to the Y-axis are captured as straight lines parallel to an y″-axis on the screen.
10 . An imaging system comprising:
an image acquisition means for acquiring an uncorrected image plane using a fisheye lens rotationally symmetric about an optical axis, an image processing means for extracting a rectilinear image plane from the uncorrected image plane; and an image display means for displaying the rectilinear image plane on a rectangular screen, wherein, if a coordinate of an image point on the rectangular screen corresponding to an object point having a coordinate (X, Y, Z) in a world coordinate system, which has a nodal point of the fisheye lens as an origin and a vertical line passing through the origin as an Y-axis, is given as (x″, y″), arbitrary straight lines in the world coordinate system appear as straight lines on the screen, a principal direction of vision for the rectilinear image plane is different from the optical axis direction of the fisheye lens, and the principal direction of vision for the rectilinear image does not pass through the center of the rectilinear image plane.
11 . An imaging system comprising:
an image acquisition means for acquiring an uncorrected image plane using a wide-angle lens rotationally symmetric about an optical axis, an image processing means for extracting a rectilinear image plane from the uncorrected image plane; and an image display means for displaying the rectilinear image plane on a rectangular screen, wherein, if a coordinate of an image point on the rectangular screen corresponding to an object point having a coordinate (X, Y, Z) in a world coordinate system, which has a nodal point of the wide-angle lens as an origin and a vertical line passing through the origin as an Y-axis, is given as (x″, y″), the rectilinear image plane is obtained from the uncorrected image plane using digital slide operation and at least one digital pan or tilt operation, and arbitrary straight lines in the world coordinate system appear as straight lines on the screen.
12 . An imaging system comprising:
an image acquisition means for acquiring an uncorrected image plane using a wide-angle lens rotationally symmetric about an optical axis, an image processing means for extracting a multiple viewpoint panoramic image plane from the uncorrected image plane; and an image display means for displaying the multiple viewpoint panoramic image plane on a rectangular screen, wherein, the multiple viewpoint panoramic image plane includes the 1 st through the n th sub rectilinear image planes laid out horizontally on the screen where n is a natural number larger than 2, an arbitrary straight line in a world coordinate system, which has a nodal point of the wide-angle lens as an origin, appears as a straight line on any of the 1 st through the n th sub rectilinear image planes, and any straight line in the world coordinate system appearing on more than two adjacent sub rectilinear image planes appears as connected line segments.
13 . The imaging system of claim 12 ,
wherein the 1 st through the n th sub rectilinear image planes have identical magnification ratios, and the optical axis of the wide-angle lens penetrates one sub rectilinear image plane among the n sub rectilinear image planes.
14 . An imaging system comprising:
an image acquisition means for acquiring an uncorrected image plane using a wide-angle lens rotationally symmetric about an optical axis, an image processing means for extracting a processed image plane from the uncorrected image plane; and an image display means for displaying the processed image plane on a rectangular screen, wherein, the processed image plane includes the 1 st through the n th sub rectilinear image planes where n is a natural number larger than 2, an arbitrary straight line in a world coordinate system having a nodal point of the wide-angle lens as an origin appears as a straight line on any of the 1 st through the n th sub rectilinear image planes.
15 . Imaging system of claim 14 ,
wherein a straight line parallel to the Y-axis in the world coordinate system and intersecting with a principal direction of vision for any sub rectilinear image plane appears as a straight line parallel to an y″-axis on the screen.
16 . The imaging system in claim 14 , further comprising an image selection means for determining parameters of the sub rectilinear image planes.
17 . The imaging system of claim 14 ,
wherein the 1 st through the n th sub rectilinear image planes have identical magnification ratios.
18 . An imaging system comprising:
an image acquisition means for acquiring an uncorrected image plane using a wide-angle lens rotationally symmetric about an optical axis, an image processing means for extracting a processed image plane from the uncorrected image plane, an image selection means for determining projection schemes and parameters of the processed image plane; and an image display means for displaying the processed image plane on a rectangular screen, wherein, the projection schemes of the processed image plane include a panoramic projection scheme and a rectilinear projection scheme, if a coordinate of an image point on the rectangular screen corresponding to an object point having a coordinate (X, Y, Z) in a world coordinate system, which has a nodal point of the wide-angle lens as an origin and a vertical line passing through the origin as an Y-axis, is given as (x″, y″), the panoramic projection scheme is a scheme wherein the lateral coordinate x″ of the image point is proportional to a horizontal incidence angle iv, herein, the horizontal incidence angle iv is an angle an incident ray originating from the object point makes with an Y-Z plane and is given as
ψ
=
tan
-
1
(
X
Z
)
,
and straight lines parallel to the Y-axis are captured as straight lines parallel to an y″-axis on the screen,
and the rectilinear projection scheme is a scheme wherein arbitrary straight lines in the world coordinate system appear as straight lines on the screen.
19 . The imaging system of claim 18 ,
wherein the wide-angle lens is a fisheye lens.
20 . The imaging system of claim 18 ,
wherein the image selection means comprise a situation awareness module.
21 . The imaging system of claim 18 ,
wherein the image selection means comprise a direction sensing means.
22 . The imaging system of claim 18 ,
wherein a straight line parallel to the Y-axis in the world coordinate system and intersecting with a principal direction of vision for a processed image plane having a rectilinear projection scheme appears as a straight line parallel to the y″-axis on the screen.
23 . The imaging system of claim 18 ,
wherein the image selection means comprise a direction sensing means, and the image acquisition means is a video camera installed on an automobile.
24 . The imaging system of claim 18 ,
wherein the image acquisition means is a camera module installed on a camera phone, the image processing means is embedded software running in the camera phone, and the image display means is a display screen of the camera phone.
25 . The imaging system of claim 18 ,
wherein the image acquisition means is a camera module installed on a camera phone, the image processing means is software running on a computer, and the image display means is a display monitor of the computer.
26 . The imaging system of claim 18 ,
wherein the image acquisition means is a PC camera, the image processing means is software running on a computer, and the image display means is a display monitor of the computer.
27 . The imaging system of claim 18 ,
wherein the image acquisition means is a digital camera, the image processing means is software running on a computer, and the image display means is a display monitor of the computer.
28 . The imaging system of claim 18 ,
wherein the image acquisition means is a camera module in a network camera, and the image processing means is embedded software running in the network camera.
29 . The imaging system of claim 18 ,
wherein the image acquisition means is a camera module on a network camera, the image processing means is software running on a computer connected to the same network as the network camera, and the image display means is a display monitor of the computer.
30 . The imaging system of claim 18 ,
wherein the image acquisition means is a camera module on a video doorphone, and the image processing means is embedded software running in the video doorphone, and the image display means is a wallpad monitor in the house.
31 . The imaging system of claim 18 ,
wherein the image acquisition means is a camera module on a video doorphone, and the image processing means is embedded software running in the video doorphone, and the image display means is display monitor of a smartphone connected to the video doorphone using a wireless Internet.
32 . The imaging system of claim 18 ,
wherein the image acquisition means is a camera module on a broadcasting camera, and the image processing means is embedded software running in the broadcasting camera.Cited by (0)
No later patents cite this yet.
References (0)
No backward citations on record.