USRE43991EExpiredUtility
Method of color filter design and color reproduction under the effect of pixel crosstalk in CMOS image sensors
Assignee: INTELLECTUAL VENTURES II LLCPriority: May 11, 2005Filed: Nov 24, 2010Granted: Feb 12, 2013
Est. expiryMay 11, 2025(expired)· nominal 20-yr term from priority
H04N 23/84H04N 25/134H10F 39/8053
68
PatentIndex Score
1
Cited by
6
References
39
Claims
Abstract
The present invention is directed at method of designing a Color Filter Arrays (CFA) for CMOS image sensors under the effects of crosstalk for optimal color reproduction. Instead of a focus on lowering crosstalk, a novel method of designing color filter spectral responses to compensate for the effect of crosstalk at the color imaging system level is proposed. As part of this method, a color reproduction model for CMOS and CCD image sensor under the effect of crosstalk is also proposed.
Claims
exact text as granted — not AI-modified1. A method of designing a image sensing system including a color filter array and an image sensor, having wherein the color filter array has spectral response curves for all color channels, for digital imaging sensors under the effect of crosstalk comprising:
determining a crosstalk characteristic function using cumulative effects of up to all pixels on a the image sensor;
modeling color reproduction based on said crosstalk characteristic function and said spectral response curves;
optimizing said spectral response curves of said color filter array wherein the shape shapes of the spectral response curves for as many as all color channels extant on the image sensor are selected to optimize the accuracy of color reproduction; and,
inputting the optimized spectral response curves of said color filter array back to the determining crosstalk characteristic function and the generating a color reproduction model steps until a predetermined metric threshold is reached or exceeded by some predetermined amount.
2. The method of claim 1 , wherein a value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an the image sensor is calculated in during said modeling color reproduction step, wherein said optimizing said spectral response curves of said color filter array step is based on includes decreasing the a difference between a predetermined threshold value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an the image sensor and said value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an the image sensor calculated in during said modeling color reproduction step.
3. The method of claim 1 , wherein said determining crosstalk characteristic functions step is achieved by direct measurement of the comprises directly measuring spectral response responses of individual pixels as a function of wavelength using a monochromator to step through a range of wavelengths to illuminate said pixels one wavelength band at a time.
4. The method of claim 3 , wherein a value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an the image sensor is calculated in during said modeling color reproduction step, wherein said optimizing said spectral response curves of said color filter array step is based on includes decreasing the a difference between a predetermined threshold value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an the image sensor and said value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an the image sensor calculated in during said modeling color reproduction step.
5. The method of claim 3 , wherein light from said monochromator is focused so that only a single pixel is illuminated at a time.
6. The method of claim 5 , wherein a value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an the image sensor is calculated in during said modeling color reproduction step, wherein said optimizing said spectral response curves of said color filter array step is based on includes decreasing the a difference between a predetermined threshold value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an the image sensor and said value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an the image sensor calculated in during said modeling color reproduction step.
7. The method of claim 3 , wherein a metal mask is applied to the digital imaging image sensor so that only one pixel is exposed at a time.
8. The method of claim 7 , wherein a value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an the image sensor is calculated in during said modeling color reproduction step, wherein said optimizing said spectral response curves of said color filter array step is based on includes decreasing the a difference between a predetermined threshold value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an the image sensor and said value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an the image sensor calculated in during said modeling color reproduction step.
9. The method of claim 1 , wherein said determining crosstalk characteristic functions step is achieved by simulation comprises simulating cumulative effects of up to all pixels on the image sensor.
10. The method of claim 9 , wherein a value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an the image sensor is calculated in during said modeling color reproduction step, wherein said optimizing said spectral response curves of said color filter array step is based on includes decreasing the a difference between a predetermined threshold value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an the image sensor and said value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an the image sensor calculated in during said modeling color reproduction step.
11. The method of claim 9 , wherein said simulation simulating the cumulative effects comprises;
constructing a process flow including all major mask steps and thermal cycles using a process simulator;
calibrating said process flow against Secondary Ion Mass Spectrometry data; and,
entering said calibrated process flow into a device simulator; and
constructing the crosstalk characteristic function using said device simulator.
12. The method of claim 11 , wherein a value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an the image sensor is calculated in during said modeling color reproduction step, wherein said optimizing said spectral response curves of said color filter array step is based on includes decreasing the a difference between a predetermined threshold value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an the image sensor and said value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an the image sensor calculated in during said modeling color reproduction step.
13. The method of claim 1 , wherein said crosstalk characteristic function is approximated as a linear function CT(λ)=a+b·λ, wherein
a
=
λ
n
·
CT
min
-
λ
l
·
CT
max
λ
max
-
λ
l
and
b
=
CT
max
-
CT
min
λ
n
-
λ
l
where the λ l and λ n represent the extremes of the a range of wavelengths, and CT min and CT max are the minimum and maximum values of crosstalk corresponding to λ l and λ n , respectively.
14. The method of claim 1 , wherein said crosstalk characteristic function is approximated as a set of piece wise linear functions, wherein each linear function is CT b (λ)=a b +b b ·λ, wherein
a
b
=
λ
bn
·
CT
minb
-
λ
bl
·
CT
maxb
λ
maxb
-
λ
bn
and
b
b
=
CT
maxb
-
CT
minb
λ
b
n
-
λ
b
l
where the λ lb λ bl and λ nb λ bn represent the extremes of the a range of wavelengths of the a b th band of the an overall wavelength range of the crosstalk characteristic function of the b th band, and CT min b and CT mab x are the minimum and maximum values of crosstalk correpsonding to λ lb λ bl and λ nb λ bn , respectively.
15. The method of claim 14 , wherein a value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an the image sensor is calculated in during said modeling color reproduction step, wherein said optimizing said spectral response curves of said color filter array step is based on includes decreasing the a difference between a predetermined threshold value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an the image sensor and said value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an the image sensor calculated in during said modeling color reproduction step.
16. The method of claim 1 , wherein said modeling color reproduction step is based on the equations comprises determining R ct (λ)=R rr (λ)+G gr (λ)+B br (λ), G ct (λ)=R rg (λ)+G gg (λ)+B bg (λ), and B ct (λ)=R rb (λ)+G gb (λ)+B bb (λ), where R ct (λ), G ct (λ), B ct (λ) are the respective composite spectral sensitivities for red, green and blue pixels in a Bayer color filter array (CFA), R rr (λ), G gg (λ), B bb (λ) are the primary spectral sensitivities components of red, green and bluish pixels respectively, R rg (λ) is the a spectral sensitivity contribution of red pixels to green pixels due to crosstalk, R rb (λ) is the a spectral sensitivity contribution of red pixels to blue pixels due to crosstalk, G gr (λ) is the a spectral sensitivity contribution of green pixels to red pixels due to crosstalk, G gb (λ) is the a spectral sensitivity contribution of green pixels to blue pixels due to crosstalk, B br (λ) is the a spectral sensitivity contribution of blue pixels to red pixels due to crosstalk and B bg (λ) is the a spectral sensitivity contribution of blue pixels to green pixels due to crosstalk.
17. The method of claim 16 , wherein a value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an image sensor is calculated in during said modeling color reproduction step, wherein said optimizing said spectral response curves of said color filter array step is based on includes decreasing the a difference between a predetermined threshold value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an image sensor and said value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an image sensor calculated in during said modeling color reproduction step.
18. The method of claim 1 , wherein said modeling color reproduction step is based on said crosstalk characteristic function, and said spectral response curves are only those available to a particular color filter array manufacturing process.
19. A method for designing a color filter array for use with an image sensor, the method comprising:
a) determining a crosstalk characteristic function of pixels of a color filter array and image sensor combination using cumulative effects of crosstalk between the pixels; b) using the crosstalk characteristic function and one or more spectral response curves of the color filter array to generate a color reproduction model; c) revising shapes of the spectral response curves for as many as all color channels extant on the image sensor to optimize color reproduction accuracy of the color reproduction model; and d) repeating operation a), operation b), and operation c) using the revised shapes of the spectral response curves determined at operation c) until a predetermined metric threshold is reached or exceeded by a predetermined amount; and e) applying resultant spectral response curves to manufacture the color filter array.
20. The method of claim 19, wherein determining the crosstalk characteristic function comprises directly measuring spectral responses of individual pixels as a function of wavelength using a monochromator to step through a range of wavelengths to illuminate the pixels one wavelength band at a time.
21. The method of claim 19, wherein revising the shapes of the spectral response curves comprises reducing a difference between a predetermined threshold value of a quality metric of the goodness of spectral sensitivities of an image sensor and the value of a quality metric of the goodness of spectral sensitivities of the image sensor used to generate the color reproduction model.
22. The method of claim 21, wherein the quality metric of the goodness is selected from a group consisting of: q-factor, μ-factor, Q st and Q sf , Figure of Merit, and Unified Measure of Goodness.
23. The method of claim 19, wherein the determining the crosstalk characteristic function comprises simulating the cumulative effects of up to all pixels on the image sensor.
24. The method of claim 19, wherein the crosstalk characteristic function is approximated as a linear function CT(λ)=a+b ·λ, wherein
a
=
λ
n
·
CT
min
-
λ
l
·
CT
max
λ
max
-
λ
l
and
b
=
CT
max
-
CT
min
λ
n
-
λ
l
where the λ l and λ n represent extremes of a range of wavelengths, and CT min and CT max are minimum and maximum values of crosstalk corresponding to λ l and λ n , respectively.
25. The method of claim 19, wherein said crosstalk characteristic function is approximated as a set of piece wise linear functions, wherein each linear function is CT b (λ)=a b +b b ·λ, wherein
a
b
=
λ
bn
·
CT
minb
-
λ
bl
·
CT
maxb
λ
maxb
-
λ
b
n
and
b
b
=
CT
maxb
-
CT
minb
λ
b
n
-
λ
b
l
where the λ bl and λ bn represent extremes of a range of wavelengths of a b th band of an overall wavelength range of the crosstalk characteristic function of the b th band, and CT min b and CT mab x are minimum and maximum values of crosstalk corresponding to λ bl and λ bn , respectively.
26. The method of claim 19, wherein the color reproduction model comprises: R ct (λ)=R rr (λ)+G gr (λ)+B br (λ), G ct (λ)=R rg (λ)+G gg (λ)+B br (λ), and B ct (λ)=R rb (λ)+G gb (λ)+B bb (λ), where R cr (λ), G cr (λ), B cr (λ) are respective composite spectral sensitivities for red, green and blue pixels in a Bayer color filter array, R rr (λ), G gg (λ), B bb (λ) are primary spectral sensitivities components of red, green and bluish pixels respectively, R rg (λ) is a spectral sensitivity contribution of red pixels to green pixels due to crosstalk, R rb (λ) is a spectral sensitivity contribution of red pixels to blue pixels due to crosstalk, G gr (λ) is a spectral sensitivity contribution of green pixels to red pixels due to crosstalk, G gb (λ) is a spectral sensitivity contribution of green pixels to blue pixels due to crosstalk, B br (λ) is a spectral sensitivity contribution of blue pixels to red pixels due to crosstalk, and B bg (λ) is a spectral sensitivity contribution of blue pixels to green pixels due to crosstalk.
27. The method of claim 30, wherein the crosstalk characteristic function is approximated as a linear function CT(λ)=a+b·λ, wherein
a
=
λ
n
·
CT
min
-
λ
l
·
CT
max
λ
max
-
λ
l
and
b
=
CT
max
-
CT
min
λ
n
-
λ
l
where the λ l and λ n represent extremes of a range of wavelengths, and CT min and CT max are minimum and maximum values of crosstalk corresponding to λ l and λ n , respectively.
28. The method of claim 30, wherein said crosstalk characteristic function is approximated as a set of piece wise linear functions, wherein each linear function is CT b (λ)=a b +b b ·λ, wherein
a
b
=
λ
bn
·
CT
minb
-
λ
bl
·
CT
maxb
λ
maxb
-
λ
b
n
and
b
b
=
CT
maxb
-
CT
minb
λ
b
n
-
λ
b
l
where λ bl and λ bn represent extremes of a range of wavelengths of a b th band of an overall wavelength range of the crosstalk characteristic function of the b th band, and CT min b and CT mab x are minimum and maximum values of crosstalk corresponding to λ bl and λ bn , respectively.
29. The method of claim 30, wherein the color reproduction model comprises: R ct (λ)=R rr (λ)+G gr (λ)+B br (λ), G ct (λ)=R rg (λ)+G gg (λ)+B br (λ), and B ct (λ)=R rb (λ)+G gb (λ)+B bb (λ), where R cr (λ), G cr (λ), B cr (λ) are respective composite spectral sensitivities for red, green and blue pixels in a Bayer color filter array, R rr (λ), G gg (λ), B bb (λ) are primary spectral sensitivities components of red, green and bluish pixels respectively, R rg (λ) is a spectral sensitivity contribution of red pixels to green pixels due to crosstalk, R rb (λ) is a spectral sensitivity contribution of red pixels to blue pixels due to crosstalk, G gr (λ) is a spectral sensitivity contribution of green pixels to red pixels due to crosstalk, G gb (λ) is a spectral sensitivity contribution of green pixels to blue pixels due to crosstalk, B br (λ) is a spectral sensitivity contribution of blue pixels to red pixels due to crosstalk, and B bg (λ) is a spectral sensitivity contribution of blue pixels to green pixels due to crosstalk.
30. A method for designing a color filter array for an image sensor comprising:
determining a crosstalk characteristic function for one or more pixels of a combined color filter array and image sensor; using the crosstalk characteristic function and a spectral response curve of the color filter array to provide a color reproduction model; generating a further spectral response curve for said color filter array, wherein the spectral response curve has a shape selected to optimize color reproduction accuracy of the color reproduction model; iteratively executing the determining, using, and generating operations until a predetermined metric threshold for the color reproduction accuracy of the color reproduction model is met or exceeded; and applying a resultant spectral response curve to manufacture the color filter array.
31. The method of claim 30, wherein the crosstalk characteristic function is determined using cumulative effects of up to all pixels on the image sensor.
32. The method of claim 30, wherein the color reproduction model is provided using the crosstalk characteristic function and a plurality of spectral response curves, wherein the plurality of spectral response curves correspond to all color channels of the image sensor.
33. The method of claim 30, wherein determining the crosstalk characteristic function comprises directly measuring spectral responses of individual pixels as a function of wavelength using a monochromator to step through a range of wavelengths to illuminate the pixels one wavelength band at a time.
34. The method of claim 30, wherein revising the shapes of the spectral response curves comprises reducing a difference between a predetermined threshold value of a quality metric of the goodness of spectral sensitivities of an image sensor and the value of a quality metric of the goodness of spectral sensitivities of an image sensor used to generate the color reproduction model.
35. The method of claim 34, wherein the quality metric of the goodness is selected from a group consisting of: q-factor, μ-factor, Q st and Q sf , Figure of Merit, and Unified Measure of Goodness.
36. The method of claim 30, wherein the determining the crosstalk characteristic function comprises simulating cumulative effects of up to all pixels on the image sensor.
37. A method of designing a color filter array for use in an image sensor system, the method comprising:
using crosstalk between pixels of an image sensor to derive a crosstalk characteristic function; generating a color reproduction model using the crosstalk characteristic function and a first color filter array design having a first spectral response; and determining whether the color reproduction model has an acceptable color reproduction quality, wherein the color reproduction model is regenerated using a second color filter array design having a second spectral response when the color reproduction model does not have the acceptable color reproduction quality.
38. The method of claim 37, further comprising determining the crosstalk characteristic function by directly measuring spectral responses of individual pixels of the image sensor as a function of wavelength using a monochromator to step through a range of wavelengths to illuminate the pixels one wavelength band at a time.
39. The method of claim 37, wherein the color reproduction quality is selected from a group of metrics consisting of: q-factor, μ-factor, Q st and Q sf , Figure of Merit, and Unified Measure of Goodness.Cited by (0)
No later patents cite this yet.
References (0)
No backward citations on record.