US2014031701A1PendingUtilityA1
Otoscanner With 3D Imaging And Structure-From-Motion
Est. expiryMar 12, 2032(~5.7 yrs left)· nominal 20-yr term from priority
Inventors:Nathanael BerglundHarris BergmanScott C. CahallJerry FosterEohan GeorgeSamuel W. HarrisGiorgos HatziliasKarol Constantine HatziliasRuizhi HongWess Eric SharpeDavid G. StitesHarry S. Strothers, Iv
A61B 1/227A61B 2090/367B33Y 80/00A61B 1/00052A61B 2034/2055A61B 5/0073A61B 5/7253A61B 5/1079A61B 5/065A61B 5/0062
53
PatentIndex Score
0
Cited by
0
References
0
Claims
Abstract
Determination of structure-from-motion that includes a scanner body having mounted upon it a tracking illumination emitter and one or more tracking illumination sensors, the tracking illumination sensors disposed upon the scanner body so as to sense reflections of tracking illumination, the scanned object characterized by an object space defined by fixed positions of tracking targets; the scanner body having mounted within it an image sensor, the scanner body characterized by a scanner space, the image sensor coupled for data communications to a data processor and a computer memory, the image sensor and the processor capturing one or more images of the scanned object; and the data processor configured so that it determines by structure-from-motion, based upon the one or more captured images and tracked positions of the probe inferred from reflections of tracking illumination, the location in object space of a feature of the scanned object.
Claims
exact text as granted — not AI-modifiedWhat is claimed is:
1 . A scanner for determination of structure-from-motion, the scanner comprising:
a scanner body having mounted upon it a tracking illumination emitter and one or more tracking illumination sensors, the tracking illumination sensors disposed upon the scanner body so as to sense reflections of tracking illumination emitted from the tracking illumination emitter and reflected from tracking targets installed at positions that are fixed relative to a scanned object, the scanned object characterized by an object space defined by the fixed positions of the tracking targets; the scanner body having mounted within it an image sensor, the scanner body characterized by a coordinate system defining a scanner space, the image sensor coupled for data communications to a data processor and a computer memory, the image sensor and the processor capturing one or more images of the scanned object; and the data processor configured so that it determines by structure-from-motion, based upon the one or more captured images and tracked positions of the probe inferred from reflections of tracking illumination, the location in object space of a feature of the scanned object.
2 . The scanner of claim 1 wherein the data processor configured so that it determines by structure-from-motion the location in object space of a feature of the scanned object further comprises the data processor configured so that it determines the location of the feature in object space as an intersection point in object space of two rays:
a first ray connecting in object space the feature and a pixel that images the feature in an image captured through the image sensor when the scanner is located at a first position in object space, and
a second ray connecting in object space the feature and a pixel that images the feature in an image captured through the image sensor when the scanner is located at a second position in object space.
3 . The scanner of claim 1 wherein the data processor configured so that it determines by structure-from-motion the location in object space of a feature of the scanned object further comprises the data processor configured so that it determines the location of the feature in object space as the solution of a triangle formed by three rays:
two side rays that connect in object space the feature and pixels that image the feature in two different images captured through the image sensor when the scanner is located at a two different positions in object space, and
a base ray connecting the two scanner positions in object space,
wherein all three angles of the triangle and the length of the base ray are determined by the processor based upon the captured images and tracked positions of the scanner inferred when the scanner is located at the two different positions, and the lengths of the side rays are determined by the processor according to the law of sines.
4 . The scanner of claim 1 wherein:
the scanner further comprises a probe mounted upon the scanner body, with the probe configured to conduct a beam of light through the probe to the feature of the scanned object so that the beam produces a dot of illumination on the feature, with reflection from the dot illuminating a pixel on the image sensor, the pixel characterized by a location in scanner space; and
the data processor configured so that it determines by structure-from-motion the location in object space of a feature of the scanned object further comprises the data processor configured so that it determines, based upon a single captured image, the location of the feature in object space by a transformation of the location of the pixel from scanner space to object space.
5 . The scanner of claim 4 wherein the transformation of the location of the pixel from scanner space to object space further comprises a transformation by a tensor that expresses the relationship between scanner space and object space according to:
[
x
′
y
′
z
′
1
]
≡
[
R
11
R
12
R
13
T
1
R
21
R
22
R
23
T
2
R
31
R
32
R
33
T
3
0
0
0
1
]
[
x
y
z
1
]
,
wherein:
the T values in the tensor express the translation of scanner space with respect to object space,
the R values express the rotation of scanner space with respect to object space,
vector x,y,z represents the pixel location in scanner space, and
vector x′,y′,z′ represents the point in object space that corresponds to the location x,y,z in scanner space.
6 . The scanner of claim 1 further comprising a probe mounted upon the scanner body, the probe comprising an imaging light source and a wide-angle lens optically coupled to the image sensor.
7 . The scanner of claim 1 wherein the scanner is an otoscanner, and the scanned object is an ear.
8 . A method of determining structure-from-motion, the method comprising:
sensing, by tracking illumination sensors disposed with a tracking illumination emitter upon a scanner body, reflections of tracking illumination emitted from a tracking illumination emitter and reflected from tracking targets installed at positions that are fixed relative to a scanned object, the scanned object characterized by an object space defined by the fixed positions of the tracking targets; capturing into computer memory by a data processor through an image sensor mounted in the scanner body one or more images of the scanned object, the scanner body characterized by a coordinate system defining a scanner space; determining by the data processor by structure-from-motion, based upon the one or more captured images and tracked positions of the probe inferred from reflections of tracking illumination, the location in object space of a feature of the scanned object.
9 . The method of claim 8 wherein determining the location in object space of a feature of the scanned object further comprises determining by the data processor the location of the feature in object space as an intersection point in object space of two rays:
a first ray connecting in object space the feature and a pixel that images the feature in an image captured through the image sensor when the scanner is located at a first position in object space, and
a second ray connecting in object space the feature and a pixel that images the feature in an image captured through the image sensor when the scanner is located at a second position in object space.
10 . The method of claim 8 wherein determining the location in object space of a feature of the scanned object further comprises determining by the data processor the location of the feature in object space as the solution of a triangle formed by three rays:
two side rays that connect in object space the feature and pixels that image the feature in two different images captured through the image sensor when the scanner is located at a two different positions in object space, and
a base ray connecting the two scanner positions in object space,
including determining by the processor all three angles of the triangle and the length of the base ray based upon the captured images and tracked positions of the scanner inferred when the scanner is located at the two different positions, and
determining by the processor the lengths of the side rays according to the law of sines.
11 . The method of claim 8 further comprising:
conducting a beam of light to the feature through a probe mounted upon the scanner body, the beam producing a dot of illumination on the feature, with reflection from the dot illuminating a pixel on the image sensor, the pixel characterized by a location in scanner space;
where determining by structure-from-motion the location in object space of a feature of the scanned object further comprises determining by the data processor, based upon a single captured image, the location of the feature in object space by a transformation of the location of the pixel from scanner space to object space.
12 . The method of claim 11 wherein the transformation of the location of the pixel from scanner space to object space further comprises a transformation by a tensor that expresses the relationship between scanner space and object space according to:
[
x
′
y
′
z
′
1
]
≡
[
R
11
R
12
R
13
T
1
R
21
R
22
R
23
T
2
R
31
R
32
R
33
T
3
0
0
0
1
]
[
x
y
z
1
]
,
wherein:
the T values in the tensor express the translation of scanner space with respect to object space,
the R values express the rotation of scanner space with respect to object space,
vector x,y,z represents the pixel location in scanner space, and
vector x′,y′,z′ represents the point in object space that corresponds to the location x,y,z in scanner space.
13 . Apparatus for determination of structure-from-motion, the apparatus comprising a data processor configured so that it determines by structure-from-motion, based upon one or more captured images of a feature of an object, the location of the feature.
14 . The apparatus of claim 13 further comprising a tracking illumination emitter and one or more tracking illumination sensors, the tracking illumination sensors placed so as to sense reflections of tracking illumination emitted from the tracking illumination emitter and reflected from tracking targets installed at positions that are fixed relative to an object, the object characterized by an object space defined by the fixed positions of the tracking targets.
15 . The apparatus of claim 13 further comprising an image sensor, the image sensor characterized by a coordinate system defining an image space, the image sensor coupled for data communications to a data processor and a computer memory, the image sensor and the processor capturing one or more images of an object.
16 . The apparatus of claim 8 further comprising the data processor configured so that it determines by structure-from-motion, based upon one or more captured images and tracked positions of the image sensor inferred from reflections of tracking illumination, the location in object space of a feature of the object.
17 . The apparatus of claim 13 wherein the data processor configured so that it determines by structure-from-motion the location in object space of a feature of the scanned object further comprises the data processor configured so that it determines the location of the feature in object space as an intersection point in object space of two rays:
a first ray connecting in object space the feature and a pixel that images the feature in an image captured through the image sensor when the scanner is located at a first position in object space, and
a second ray connecting in object space the feature and a pixel that images the feature in an image captured through the image sensor when the scanner is located at a second position in object space.
18 . The apparatus of claim 13 wherein the data processor configured so that it determines by structure-from-motion the location in object space of a feature of the scanned object further comprises the data processor configured so that it determines the location of the feature in object space as the solution of a triangle formed by three rays:
two side rays that connect in object space the feature and pixels that image the feature in two different images captured through the image sensor when the scanner is located at a two different positions in object space, and
a base ray connecting the two scanner positions in object space,
wherein all three angles of the triangle and the length of the base ray are determined by the processor based upon the captured images and tracked positions of the scanner inferred when the scanner is located at the two different positions, and the lengths of the side rays are determined by the processor according to the law of sines.
19 . The apparatus of claim 13 wherein:
the apparatus further comprises a scanner body, a probe mounted upon the scanner body, and an image sensor, with the probe configured to conduct a beam of light through the probe to the feature of the scanned object so that the beam produces a dot of illumination on the feature, with reflection from the dot illuminating a pixel on the image sensor, the pixel characterized by a location in scanner space; and
the data processor configured so that it determines by structure-from-motion the location in object space of a feature of the scanned object further comprises the data processor configured so that it determines, based upon a single captured image, the location of the feature in object space by a transformation of the location of the pixel from scanner space to object space.
20 . The apparatus of claim 19 wherein the transformation of the location of the pixel from scanner space to object space further comprises a transformation by a tensor that expresses the relationship between scanner space and object space according to:
[
x
′
y
′
z
′
1
]
≡
[
R
11
R
12
R
13
T
1
R
21
R
22
R
23
T
2
R
31
R
32
R
33
T
3
0
0
0
1
]
[
x
y
z
1
]
,
wherein:
the T values in the tensor express the translation of scanner space with respect to object space,
the R values express the rotation of scanner space with respect to object space,
vector x,y,z represents the pixel location in scanner space, and
vector x′,y′,z′ represents the point in object space that corresponds to the location x,y,z in scanner space.Cited by (0)
No later patents cite this yet.
References (0)
No backward citations on record.