US7565029B2ExpiredUtilityPatentIndex 89
Method for determining camera position from two-dimensional images that form a panorama
Est. expiryJul 8, 2025(expired)· nominal 20-yr term from priority
G06T 2200/32G06T 2207/10012G06T 7/80
89
PatentIndex Score
39
Cited by
15
References
14
Claims
Abstract
A method of estimating three-dimensional camera position information from a series of two-dimensional images that form a panorama employs common features in adjoining image pairs in the series to estimate a transform between the images in the pairs. The common features are subsequently employed to adjust an estimated rotational component of each transform by reducing error between coordinates corresponding to the common features in three-dimensional space in image pairs, on a pair-by-pair basis. A global optimization of the position estimation, used for long sequences of images such as 360 degree panoramas, refines the estimates of the rotational and focal length components of the transforms by concurrently reducing error between all 3D common feature coordinates for all adjoining pairs.
Claims
exact text as granted — not AI-modified1. A method of estimating camera position information from a series of source images that form a panorama, the method comprising:
using a processor to carry out the steps of;
estimating a transform between adjoining image pairs in said series based on common features within said adjoining image pairs;
determining a rotational component for each transform and three-dimensional coordinates for said common features;
adjusting each of the rotational components to reduce errors between corresponding ones of said coordinates in adjoining image pairs; and
calculating focal lengths based on the adjusted rotational components;
wherein said determining is done on the basis of an initial focal length;
in the event that predetermined conditions are satisfied, concurrently refining the focal lengths and rotational components to globally reduce errors between corresponding ones of said coordinates in all image pairs;
prior to said refining, adjusting a position of each of said images to an average latitude of all images in said series;
setting the y-component of the x-axis of each image to zero; and
setting the y-axis of each image to the cross-product of said x-axis and the respective image normal.
2. The method of claim 1 , wherein said determining is done on the basis that said source images are evenly spaced across said panorama.
3. The method of claim 1 , wherein said common features are matching corners.
4. The method of claim 3 , wherein said transform is a projective transform.
5. The method of claim 1 , wherein said initial estimate of focal length for each image is a respective image diagonal length calculated according to:
Diagonal_Length
=
(
IMAGE_Width
)
2
+
(
IMAGE_Height
)
2
6. The method of claim 1 , wherein three-dimensional image parameters for use during said determining are calculated based on said initial estimate of focal length.
7. The method of claim 6 , wherein said three-dimensional image parameters comprise image center point, image vertices, image normal, x-axis and y-axis parameters.
8. A method of estimating camera position information from a series of source images that form a panorama, the method comprising:
using a processor to carry out the steps of;
estimating a transform between adjoining image pairs in said series based on common features within said adjoining image pairs;
determining a rotational component for each transform and three-dimensional coordinates for said common features;
adjusting each of the rotational components to reduce errors between corresponding ones of said coordinates in adjoining image pairs; and
calculating focal lengths based on the adjusted rotational components;
wherein in the event that predetermined conditions are satisfied, concurrently refining the focal lengths and rotational components to globally reduce errors between corresponding ones of said coordinates in all image pairs;
prior to said refining, adjusting a position of each of said images to an average latitude of all images in said series;
setting the v-component of the x-axis of each image to zero; and
setting the v-axis of each image to the cross-product of said x-axis and the respective image normal;
wherein said determining is done on the basis of an initial focal length;
prior to said adjusting, placing a reference image onto an equator of a three-dimensional model, at a distance equal to a respective initial estimate of focal length from an origin of said model; and
sequentially placing adjoining images into said model based on respective initial estimates of focal length and rotational component and identifying rays from said origin to corresponding ones of said coordinates in adjoining images;
wherein said adjusting comprises minimizing error in direction between corresponding ones of said rays.
9. The method of claim 8 , wherein said error in direction is the Euclidean distance between corresponding ones of said coordinates, according to:
q
(
x
r
y
r
z
r
)
=
(
x
c
y
c
z
c
)
+
s
(
x
A
y
A
z
A
)
+
t
(
x
B
y
B
z
B
)
where (X r y r Z r ) T is the normalized vector of a ray to a coordinate in a first adjoining image in one of said pairs;
(x c y c z c ) T is the three-dimensional center of a second adjoining image in said one pair;
(X A Y A Z A ) T is the x-axis of said second adjoining image; and
(XB y z8 )T is the y-axis of said second adjoining image.
10. The method of claim 9 , wherein center points of all images in a common row are assumed to have a common latitude.
11. The method of claim 10 , wherein said placing and adjusting steps are performed iteratively a predetermined number of times.
12. The method of claim 1 , wherein said predetermined conditions comprise at least one of:
said series of source images form a 360 degree panorama;
said series of source images form greater than one row with an extent greater than one; and
said series of source images form greater than one column with an extent greater than one.
13. The method of 1 , wherein said refining comprises correcting focal lengths, and rotation angles for each of three-dimensions.
14. An apparatus for estimating three dimensional camera position information from a set of two dimensional images forming a panorama, the device comprising:
a transform estimator estimating a transform between adjoining image pairs in said series based on common features within said adjoining image pairs;
a rotational component determinor determining a rotational component for each transform and three-dimensional coordinates for said common features;
an adjustor adjusting each of the rotational components to reduce errors between corresponding ones of said coordinates in adjoining image pairs; and
a focal length calculator calculating focal lengths based on the adjusted rotational components
wherein said determining is done on the basis of an initial focal length;
in the event that predetermined conditions are satisfied, concurrently refining the focal lengths and rotational components to globally reduce errors between corresponding ones of said coordinates in all image pairs;
prior to said refining, adjusting a position of each of said images to an average latitude of all images in said series;
setting the y-component of the x-axis of each image to zero; and
setting the v-axis of each image to the cross-product of said x-axis and the respective image normal.Cited by (0)
No later patents cite this yet.
References (0)
No backward citations on record.