US2004008269A1PendingUtilityA1
System and method for providing multi-sensor super-resolution
Priority: Nov 30, 2001Filed: Nov 30, 2002Published: Jan 15, 2004
Est. expiryNov 30, 2021(expired)· nominal 20-yr term from priority
G06T 3/4053
41
PatentIndex Score
0
Cited by
0
References
0
Claims
Abstract
A super-resolution enhanced image generating system is described for generating a super-resolution-enhanced image from an image of a scene, identified as image g 0 , comprising a base image and at least one other image g i , the system comprising an initial super-resolution enhanced image generator, an image projector module and a super-resolution enhanced image estimate update generator module. The initial super-resolution enhanced image generator module is configured to use the image g 0 to generate a super-resolution enhanced image estimate. The image projector module is configured to selectively use a warping, a blurring and/or a decimation operator associated with the image g i to generate a projected super-resolution enhanced image estimate. The super-resolution enhanced image estimate update generator module is configured to use the input image g i and the super-resolution enhanced image estimate to generate an updated super-resolution enhanced image estimate.
Claims
exact text as granted — not AI-modifiedWhat is claimed as new and desired to be secured by Letters Patent of the United States is:
1 . A super-resolution enhanced image generating system for generating a super-resolution-enhanced image from an image of a scene, identified as image g 0 , comprising a base image and at least one other image g i the system comprising:
A an initial super-resolution enhanced image generator module configured to use the image go to generate a super-resolution enhanced image estimate; B. an image projector module configured to selectively use a warping, a blurring and/or a decimation operator associated with the image g i to generate a projected super-resolution enhanced image estimate; and C. a super-resolution enhanced image estimate update generator module configured to use the input image g i and the super-resolution enhanced image estimate to generate an updated super-resolution enhanced image estimate.
2 . A super-resolution enhanced image generating system as defined in claim 1 , the system being configured to generate the super-resolution enhanced image in relation to a plurality of other images identified as images
{
g
i
}
i
=
1
n
,
the system further comprising an image selector module configured to select one of the images
{
g
i
}
i
=
1
n
,
the image projector module being configured to selectively use the warping, a blurring and/or a decimation operator associated with the image g, selected by the image selector module to generate a projected super-resolution enhanced image estimate, the super-resolution enhanced image estimate update generator module being configured to use the selected input image g i and the super-resolution enhanced image estimate to generate an updated super-resolution enhanced image.
3 . A super-resolution enhanced image generating system as defined in claim 2 further comprising a control module configured to control the image selector module, image projector module, and the super-resolution enhanced image update generator to operate in successive iterations using respective ones of the images
{
g
i
}
i
=
1
n
,
in each iteration, the image projector module using the updated super-resolution enhanced image generated during the previous iteration as the super-resolution enhanced image estimate for the respective iteration.
4 . A super-resolution enhanced image generating system as defined in claim 2 in which the super-resolution enhanced image estimate update generator is configured to generate at least one parameter in connection with a least squares estimate between the selected input image and the projected super-resolution enhanced image estimate.
5 . A super-resolution enhanced image generating system as defined in claim 4 in which the super resolution enhanced image estimate update generator is configured to generate the super-resolution enhanced image update estimate, for the selected input image
{
g
i
}
i
=
1
n
,
in accordance with a virtual input image ĝ i defined by
g i ( x,y )= ag i ( x,y )+ b
where “a” and “b” are parameters associated with respective pixel location (x,y) defined by
[
a
b
]
=
[
p
_
i
2
(
x
,
y
)
p
_
i
(
x
,
y
)
p
_
i
(
x
,
y
)
1
]
-
1
[
p
i
g
i
_
(
x
,
y
)
g
i
_
(
x
,
y
)
]
,
and where {overscore (p i )}(x,y), {overscore (g i )}(x,y), {overscore (p i g i )}(x,y), {overscore (p i 2 )}(x,y), and {overscore (g i 2 )}(x,y) are weighted averages around the pixel at coordinates (x,y) in the respective image projected super-resolution enhanced image estimate p i or the input image g i .
6 . A super-resolution enhanced image generating system as defined in claim 5 in which the super resolution enhanced image estimate update generator is configured to generate the weighted averages in accordance with
p
_
i
(
x
,
y
)
=
∑
-
k
≤
u
,
v
≤
k
w
(
u
,
v
)
p
i
(
x
+
u
,
y
+
v
)
p
i
g
i
_
(
x
,
y
)
=
∑
-
k
≤
u
,
v
≤
k
w
(
u
,
v
)
p
i
(
x
+
u
,
y
+
v
)
g
i
(
x
+
u
,
y
+
v
)
p
_
i
2
(
x
,
y
)
=
∑
-
k
≤
u
,
v
≤
k
w
(
u
,
v
)
(
p
i
(
x
+
u
,
y
+
v
)
)
2
,
and so forth, where “w(u,v)” is a weighting kernel and “(u,v)” defines a region R around the respective pixel at location “(x,y).”
7 . A super-resolution enhanced image generating system as defined in claim 5 in which the super resolution enhanced image estimate update generator is further configured to generate the super-resolution enhanced image update estimate, for the selected input image
{
g
i
}
i
=
1
n
,
in accordance with a cross-correlation value c(x,y) generated for respective a pixel at location (x,y) in the respective image defined by
c
(
x
,
y
)
=
p
i
g
i
_
(
x
,
y
)
-
p
i
_
(
x
,
y
)
g
i
_
(
x
,
y
)
(
p
i
2
_
(
x
,
y
)
-
(
p
i
_
(
x
,
y
)
)
2
)
(
g
i
2
_
(
x
,
y
)
-
(
g
_
i
(
x
,
y
)
)
2
)
,
and a selected threshold value.
8 . A method of generating a super-resolution-enhanced image from an image of a scene, identified as image go, comprising a base image and at least one other image g i , the system comprising:
A an initial super-resolution enhanced image generation step of using the image g 0 to generate a super-resolution enhanced image estimate; B. an image projection step of using a warping, a blurring and/or a decimation operator associated with the image g i to generate a projected super-resolution enhanced image estimate; and C. a super-resolution enhanced image estimate update generator module configured to use the input image g i and the super-resolution enhanced image estimate to generate an updated super-resolution enhanced image estimate.
9 . A method as defined in claim 8 , the method being configured to generate the super-resolution enhanced image in relation to a plurality of other images identified as images
{
g
i
}
i
=
1
n
,
the method further comprising an image selection step of selecting one of the images
{
g
i
}
i
=
1
n
,
the image projection step including the step of selectively using the warping, a blurring and/or a decimation operator associated with the selected image g i to generate a projected super-resolution enhanced image estimate, the super-resolution enhanced image estimate update generation step including the step of using use the selected input image g i and the super-resolution enhanced image estimate to generate an updated super-resolution enhanced image.
10 . A method as defined in claim 9 further comprising a step of enabling the image selection step, the image projection step, and the super-resolution enhanced image update generation step to be performed in successive iterations using respective ones of the images
{
g
i
}
i
=
1
n
,
in each iteration, the image projection step including the step of using the updated super-resolution enhanced image generated during the previous iteration as the super-resolution enhanced image estimate for the respective iteration.
11 . A super-resolution enhanced image generating method as defined in claim 10 in which the super-resolution enhanced image estimate update step includes the step of generating at least one parameter in connection with a least squares estimate between the selected input image and the projected super-resolution enhanced image estimate.
12 . A super-resolution enhanced image generating method as defined in claim 11 in which the super resolution enhanced image estimate update generation step includes the step of generating the super-resolution enhanced image update estimate, for the selected input image
{
g
i
}
i
=
1
n
,
in accordance with a virtual input image ĝ i defined by
ĝ i ( x,y )= ag i ( x,y )+ b
where “a” and “b” are parameters associated with respective pixel location (x,y) defined by
[
a
b
]
=
[
p
_
i
2
(
x
,
y
)
p
_
i
(
x
,
y
)
p
_
i
(
x
,
y
)
1
]
-
1
[
p
i
g
i
_
(
x
,
y
)
g
_
i
(
x
,
y
)
]
and where
p
_
i
(
x
,
y
)
,
g
_
i
(
x
,
y
)
,
p
i
g
i
_
(
x
,
y
)
,
p
i
2
_
(
x
,
y
)
,
and
g
i
2
_
(
x
,
y
)
are weighted averages around the pixel at coordinates (x,y) in the respective image p i or g i .
13 . A super-resolution enhanced image generating method as defined in claim 12 in which the super resolution enhanced image estimate update generator is configured to generate the weighted averages in accordance with
p
i
_
(
x
,
y
)
=
∑
-
k
≤
u
,
v
≤
k
w
(
u
,
v
)
p
i
(
x
+
u
,
y
+
v
)
p
i
g
i
_
(
x
,
y
)
=
∑
-
k
≤
u
,
v
≤
k
w
(
u
,
v
)
p
i
(
x
+
u
,
y
+
v
)
g
i
(
x
+
u
,
y
+
v
)
p
i
2
_
(
x
,
y
)
=
∑
-
k
≤
u
,
v
≤
k
w
(
u
,
v
)
(
p
i
(
x
+
u
,
y
+
v
)
)
2
,
and so forth, where “w(u,v)” is a weighting kernel and “(u,v)” defines a region R around the respective pixel at location “(x,y).”
14 . A super-resolution enhanced image generating method as defined in claim 12 in which the super resolution enhanced image estimate update generation step includes the step of generating the super-resolution enhanced image update estimate, for the selected input image {g i } i=1 n , in accordance with a cross-correlation value c(x,y) generated for respective a pixel at location (x,y) in the respective image defined by
c
(
x
,
y
)
=
p
i
g
i
_
(
x
,
y
)
-
p
_
i
(
x
,
y
)
g
_
i
(
x
,
y
)
(
p
i
2
_
(
x
,
y
)
-
(
p
i
_
(
x
,
y
)
)
2
)
(
g
i
2
_
(
x
,
y
)
-
(
g
_
i
(
x
,
y
)
)
2
)
,
and a selected threshold value.
15 . A computer program product for use in connection with a computer to provide a super-resolution enhanced image generating system for generating a super-resolution-enhanced image from an image of a scene, identified as image g 0 , comprising a base image and at least one other image g i , the computer program product comprising a computer-readable medium having encoded thereon:
A an initial super-resolution enhanced image generator module configured to enable the computer use the image go to generate a super-resolution enhanced image estimate; B. an image projector module configured to enable the computer to selectively use a warping, a blurring and/or a decimation operator associated with the image g i to generate a projected super-resolution enhanced image estimate; and C. a super-resolution enhanced image estimate update generator module configured to enable the computer to use the input image g, and the super-resolution enhanced image estimate to generate an updated super-resolution enhanced image estimate.
16 . A computer program product as defined in claim 15 , the system being configured to generate the super-resolution enhanced image in relation to a plurality of other images identified as images
{
g
i
}
i
=
1
n
,
the computer program product further comprising an image selector module configured to enable the computer to select one of the images
{
g
i
}
i
=
1
n
,
the image projector module being configured to enable the computer to selectively use the warping, a blurring and/or a decimation operator associated with the image g i selected by the image selector module to generate a projected super-resolution enhanced image estimate, the super-resolution enhanced image estimate update generator module being configured to enable the computer to use the selected input image g i and the super-resolution enhanced image estimate to generate an updated super-resolution enhanced image.
17 . A computer program product as defined in claim 2 further comprising a control module configured to enable the computer to control processing of the image selector module, image projector module, and the super-resolution enhanced image update generator to operate in successive iterations using respective ones of the images
{
g
i
}
i
=
1
n
,
in each iteration, the image projector module being configured to use the updated super-resolution enhanced image generated during the previous iteration as the super-resolution enhanced image estimate for the respective iteration.
18 . A computer program product as defined in claim 16 in which the super resolution enhanced image estimate update generator is configured to enable the computer to generate at least one parameter in connection with a least squares estimate between the selected input image and the projected super-resolution enhanced image estimate.
19 . A computer program product as defined in claim 18 in which the super resolution enhanced image estimate update generator is configured to enable the computer to generate the super-resolution enhanced image update estimate, for the selected input image
{
g
i
}
i
=
1
n
,
in accordance with a virtual input image ĝ i defined by
ĝ i ( x,y )= ag i ( x,y )+ b
where “a” and “b” are parameters associated with respective pixel location (x,y) defined by
[
a
b
]
=
[
p
_
i
2
(
x
,
y
)
p
_
i
(
x
,
y
)
p
_
i
(
x
,
y
)
1
]
-
1
[
p
i
g
i
_
(
x
,
y
)
g
i
_
(
x
,
y
)
]
,
and where {overscore (p i )}(x,y), {overscore (g i )}(x,y), {overscore (p i g i )}(x,y), {overscore (p i 2 )}(x, y), and {overscore (g i 2 )}(x,y) are weighted averages around the pixel at coordinates (x,y) in the respective image projected super-resolution enhanced image estimate p i or the input image g i .
20 . A computer program product as defined in claim 19 in which the super resolution enhanced image estimate update generator is configured to enable the computer to generate the weighted averages in accordance with
p
_
i
(
x
,
y
)
=
∑
-
k
≤
u
,
v
≤
k
w
(
u
,
v
)
p
i
(
x
+
u
,
y
+
v
)
p
i
g
i
_
(
x
,
y
)
=
∑
-
k
≤
u
,
v
≤
k
w
(
u
,
v
)
p
i
(
x
+
u
,
y
+
v
)
g
i
(
x
+
u
,
y
+
v
)
p
_
i
2
(
x
,
y
)
=
∑
-
k
≤
u
,
v
≤
k
w
(
u
,
v
)
(
p
i
(
x
+
u
,
y
+
v
)
)
2
,
and so forth, where “w(u,v)” is a weighting kernel and “(u,v)” defines a region R around the respective pixel at location “(x,y).”
21 . A computer program product as defined in claim 19 in which the super resolution enhanced image estimate update generator is further configured to enable the computer to generate the super-resolution enhanced image update estimate, for the selected input image
{
g
i
}
i
=
1
n
,
in accordance with a cross-correlation value c(x,y) generated for respective a pixel at location (x,y) in the respective image defined by
c
(
x
,
y
)
=
p
i
g
i
_
(
x
,
y
)
-
p
_
i
(
x
,
y
)
g
_
i
(
x
,
y
)
(
p
i
2
_
(
x
,
y
)
-
(
p
i
_
(
x
,
y
)
)
2
)
(
g
i
2
_
(
x
,
y
)
-
(
g
_
i
(
x
,
y
)
)
2
)
,
and a selected threshold value.Join the waitlist — get patent alerts
Track US2004008269A1 — get alerts on status changes and closely related new filings.
We store only your email — no account needed. See our privacy policy.