Image processing device, imaging device, image processing method, and image processing program

Abstract

Disclosed are an image processing device, an imaging device, an image processing method, and an image processing program capable of, when recovering a deteriorated image due to a point spread function of an optical system, suppressing the occurrence of artifact and color gradation and achieving reduction in computational costs. The image processing device includes a frequency recovery processing unit which subjects image data acquired from an imaging element by capturing an object image using an optical system to frequency recovery processing using a frequency recovery filter based on a point spread function of the optical system, a gradation correction processing unit which subjects image data subjected to the frequency recovery processing to nonlinear gradation correction, and a phase recovery processing unit which subjects image data subjected to the gradation correction to phase recovery processing using a phase recovery filter based on the point spread function of the optical system.

What is claimed is: 
     
       1. An image processing device comprising:
 a frequency recovery processing circuit which subjects image data acquired from an imaging element by capturing an object image using an optical system to frequency recovery processing using a frequency recovery filter based on a point spread function of the optical system; 
 a gradation correction processing circuit which subjects image data subjected to the frequency recovery processing to nonlinear gradation correction; and 
 a phase recovery processing circuit which subjects image data subjected to the gradation correction to phase recovery processing using a phase recovery filter based on the point spread function of the optical system. 
 
     
     
       2. The image processing device according to  claim 1 , further comprising:
 a storage unit which stores the frequency recovery filter and the phase recovery filter, 
 wherein the frequency recovery processing circuit reads the frequency recovery filter from the storage unit and uses the frequency recovery filter in the frequency recovery processing, and 
 the phase recovery processing circuit reads the phase recovery filter from the storage unit and uses the phase recovery filter in the phase recovery processing. 
 
     
     
       3. The image processing device according to  claim 1 , further comprising:
 a storage unit which stores the point spread function of the optical system, an optical transfer function obtained by Fourier-transforming the point spread function, or a modulation transfer function indicating an amplitude component of the optical transfer function and a phase transfer function indicating a phase component of the optical transfer function, 
 wherein the frequency recovery processing circuit reads the point spread function, the optical transfer function, or the modulation transfer function from the storage unit, generates the frequency recovery filter, and uses the generated frequency recovery filter in the frequency recovery processing, and 
 the phase recovery processing circuit reads the point spread function, the optical transfer function, or the phase transfer function from the storage unit, generates the phase recovery filter, and uses the generated phase recovery filter in the phase recovery processing. 
 
     
     
       4. The image processing device according to  claim 1 ,
 wherein the frequency recovery processing circuit subjects image data acquired from the imaging element, which is image data for each color channel, to frequency recovery processing using a frequency recovery filter, and 
 the phase recovery processing circuit subjects the image data subjected to the gradation correction, which is image data for each color channel, to phase recovery processing using a phase recovery filter. 
 
     
     
       5. The image processing device according to  claim 2 ,
 wherein the frequency recovery processing circuit subjects image data acquired from the imaging element, which is image data for each color channel, to frequency recovery processing using a frequency recovery filter, and 
 the phase recovery processing circuit subjects the image data subjected to the gradation correction, which is image data for each color channel, to phase recovery processing using a phase recovery filter. 
 
     
     
       6. The image processing device according to  claim 3 ,
 wherein the frequency recovery processing circuit subjects image data acquired from the imaging element, which is image data for each color channel, to frequency recovery processing using a frequency recovery filter, and 
 the phase recovery processing circuit subjects the image data subjected to the gradation correction, which is image data for each color channel, to phase recovery processing using a phase recovery filter. 
 
     
     
       7. The image processing device according to  claim 1 ,
 wherein the frequency recovery processing circuit subjects image data acquired from the imaging element, which is image data for each color channel, to frequency recovery processing using the frequency recovery filter, and 
 the phase recovery processing circuit subjects image data subjected to gradation correction by the gradation correction processing circuit, which is image data indicating a brightness component generated from image data for each color channel, to phase recovery processing using the phase recovery filter. 
 
     
     
       8. The image processing device according to  claim 2 ,
 wherein the frequency recovery processing circuit subjects image data acquired from the imaging element, which is image data for each color channel, to frequency recovery processing using the frequency recovery filter, and 
 the phase recovery processing circuit subjects image data subjected to gradation correction by the gradation correction processing circuit, which is image data indicating a brightness component generated from image data for each color channel, to phase recovery processing using the phase recovery filter. 
 
     
     
       9. The image processing device according to  claim 3 ,
 wherein the frequency recovery processing circuit subjects image data acquired from the imaging element, which is image data for each color channel, to frequency recovery processing using the frequency recovery filter, and 
 the phase recovery processing circuit subjects image data subjected to gradation correction by the gradation correction processing circuit, which is image data indicating a brightness component generated from image data for each color channel, to phase recovery processing using the phase recovery filter. 
 
     
     
       10. The image processing device according to  claim 1 ,
 wherein the frequency recovery processing circuit subjects image data acquired from the imaging element, which is image data indicating a brightness component generated from image data for each color channel, to frequency recovery processing using the frequency recovery filter, and 
 the phase recovery processing circuit subjects the image data subjected to the gradation correction, which is image data indicating a brightness component generated from image data for each color channel, to phase recovery processing using the phase recovery filter. 
 
     
     
       11. The image processing device according to  claim 2 ,
 wherein the frequency recovery processing circuit subjects image data acquired from the imaging element, which is image data indicating a brightness component generated from image data for each color channel, to frequency recovery processing using the frequency recovery filter, and 
 the phase recovery processing circuit subjects the image data subjected to the gradation correction, which is image data indicating a brightness component generated from image data for each color channel, to phase recovery processing using the phase recovery filter. 
 
     
     
       12. The image processing device according to  claim 3 ,
 wherein the frequency recovery processing circuit subjects image data acquired from the imaging element, which is image data indicating a brightness component generated from image data for each color channel, to frequency recovery processing using the frequency recovery filter, and 
 the phase recovery processing circuit subjects the image data subjected to the gradation correction, which is image data indicating a brightness component generated from image data for each color channel, to phase recovery processing using the phase recovery filter. 
 
     
     
       13. The image processing device according to  claim 10 , further comprising:
 a brightness data generation circuit which generates brightness data indicating a brightness component from image data for each color channel acquired from the imaging element, 
 wherein the frequency recovery processing circuit subjects brightness data generated by the brightness data generation circuit to frequency recovery processing using the frequency recovery filter, 
 the gradation correction processing circuit subjects the brightness data subjected to the frequency recovery processing to nonlinear gradation correction, and 
 the phase recovery processing circuit subjects the brightness data subjected to the gradation correction to phase recovery processing using the phase recovery filter. 
 
     
     
       14. The image processing device according to  claim 11 , further comprising:
 a brightness data generation circuit which generates brightness data indicating a brightness component from image data for each color channel acquired from the imaging element, 
 wherein the frequency recovery processing circuit subjects brightness data generated by the brightness data generation circuit to frequency recovery processing using the frequency recovery filter, 
 the gradation correction processing circuit subjects the brightness data subjected to the frequency recovery processing to nonlinear gradation correction, and 
 the phase recovery processing circuit subjects the brightness data subjected to the gradation correction to phase recovery processing using the phase recovery filter. 
 
     
     
       15. The image processing device according to  claim 1 ,
 wherein the gradation correction processing circuit is a gamma correction processing circuit which subjects the image data to gradation correction by logarithmic processing. 
 
     
     
       16. The image processing device according to  claim 1 ,
 wherein the bit length of the image data subjected to frequency recovery processing by the frequency recovery processing circuit is greater than the bit length of the image data subjected to phase recovery processing by the phase recovery processing circuit. 
 
     
     
       17. The image processing device according to  claim 1 ,
 wherein the optical system has a lens circuit which enlarges a depth of field by modulating a phase. 
 
     
     
       18. An imaging device comprising:
 an imaging element which outputs image data by capturing an object image using an optical system; and 
 the image processing device according to  claim 1 . 
 
     
     
       19. An image processing method using the image processing device according to  claim 1 , comprising:
 a step of subjecting image data acquired from an imaging element by capturing an object image using an optical system to frequency recovery processing using a frequency recovery filter based on a point spread function of the optical system; 
 a step of subjecting image data subjected to the frequency recovery processing to nonlinear gradation correction; and 
 a step of subjecting image data subjected to the gradation correction to phase recovery processing using a phase recovery filter based on the point spread function of the optical system. 
 
     
     
       20. A non-transitory computer readable recording medium storing an image processing program which causes a computer to execute:
 a step of subjecting image data acquired from an imaging element by capturing an object image using an optical system to frequency recovery processing using a frequency recovery filter based on a point spread function of the optical system; 
 a step of subjecting image data subjected to the frequency recovery processing to nonlinear gradation correction; and 
 a step of subjecting image data subjected to the gradation correction to phase recovery processing using a phase recovery filter based on the point spread function of the optical system.