Computational imaging using variable optical transfer function

Abstract

In selected embodiments, improved image restoration is realized using extensions of Wiener filtering combined with multiple image captures acquired after simple, fast reconfigurations of an optical imaging system. These reconfigurations may yield distinct OTF responses for each capture. The optical imaging system may reduce fabrication cost, power consumption, and/or system weight/volume by correcting significant optical aberrations. The system may be configured to perform independent correction of fields within the total field-of-regard. The system may also be configured to perform independent correction of different spectral bands.

What is claimed is: 
     
         1 . An imaging method, comprising:
 capturing a plurality of M captured images of an object with an optical system, the optical system comprising a configurable optical component and an image sensor, the configurable optical component being capable of being configured in a plurality of configurations, wherein each captured image of the plurality of M images is captured on the image sensor with the configurable optical component being in a different corresponding configuration of the plurality of configurations and with a different optical transfer function between the object and the image sensor;   transforming each captured image of the plurality of M captured images using a selected spatial transform to obtain a corresponding transformed captured image, thereby obtaining a plurality of M transformed captured images;   weighting each transformed captured image of the plurality of M transformed captured images by a weighting coefficient A m  computed using the formula   
       
         
           
             
               
                 
                   A 
                   m 
                 
                 = 
                 
                   
                     R 
                     m 
                     * 
                   
                   
                     
                       
                         S 
                         noise 
                       
                       
                         S 
                         obj 
                       
                     
                     + 
                     
                       
                         ∑ 
                         
                           m 
                           = 
                           1 
                         
                         M 
                       
                        
                       
                         
                            
                           
                             R 
                             m 
                           
                            
                         
                         2 
                       
                     
                   
                 
               
               , 
             
           
         
       
       wherein R m  is the optical transfer function of the optical system in the configuration corresponding to the captured image of the plurality of M captured images from which said each transformed captured image of the plurality of M transformed captured images was obtained, R* m  is the complex conjugate of R m , S Noise  is the average power spectral density of the noise projection of the object, and S Obj  is the average power spectral density of the noise-free projection of the object, resulting in a weighted image corresponding to said transformed captured image of the plurality of M transformed captured images, thereby obtaining a plurality of M weighted images;
 summing the weighted images of the plurality of M weighted images to obtain a summed transformed image; 
 inverse-transforming the summed transformed image using inverse transform of the selected spatial transform to obtain a processed image; and 
 outputting the processed image, the step of outputting comprising storing the processed image in memory, displaying the processed image, or transmitting the processed image electronically over a network; 
 wherein: 
 each of the steps of capturing, transforming, weighting, summing, and inverse transforming is performed at least in part by at least one processor of at least one computer system; and 
 one or more zeroes of the optical transfer function of the optical system differ for at least two configurations of the plurality of different configurations corresponding to at least two images of the plurality of M captured images. 
 
     
     
         2 . The imaging method of  claim 1 , wherein:
 the selected spatial transform is a spatial Fourier Transform, and the inverse transform is an inverse spatial Fourier Transform;   the configurable optical component comprises a deformable mirror; and   the method further comprises configuring the deformable mirror in the plurality of different configurations using one or more control parameters of the deformable mirror.   
     
     
         3 . An apparatus for processing images, the apparatus comprising:
 an optical system comprising a configurable optical component and an image sensor, the configurable optical component being capable of being configured in a plurality of different configurations, each different configuration of the plurality of different configurations corresponding to a different optical transfer function; and   at least one processor, wherein the at least one processor is coupled to the optical system to enable the at least one processor to control configuration of the configurable component and to capture images in a focal plane of the optical system, and wherein the at least one processor is configured to execute program code instructions to cause the apparatus to perform steps comprising:
 capturing a plurality of M captured images of an object through the optical system, wherein each captured image of the plurality of M captured images is captured with the configurable optical component being in a different corresponding configuration of the plurality of configurations and with a different optical transfer function between the object and the image sensor; 
 transforming each captured image of the plurality of M captured images using a selected spatial transform to obtain a corresponding transformed captured image, thereby obtaining a plurality of M transformed captured images; 
 weighting each transformed captured image of the plurality of M transformed captured images by a weighting coefficient A m  computed using the rormula 
   
       
         
           
             
               
                 
                   A 
                   m 
                 
                 = 
                 
                   
                     R 
                     m 
                     * 
                   
                   
                     
                       
                         S 
                         noise 
                       
                       
                         S 
                         obj 
                       
                     
                     + 
                     
                       
                         ∑ 
                         
                           m 
                           = 
                           1 
                         
                         M 
                       
                        
                       
                         
                            
                           
                             R 
                             m 
                           
                            
                         
                         2 
                       
                     
                   
                 
               
               , 
             
           
         
       
       wherein R m  is the optical transfer function of the optical system in the configuration corresponding to the captured image of the plurality of M captured images from which said each transformed captured image of the plurality of M transformed captured images was obtained, R* m  is the complex conjugate of R m , S Noise  is the average power spectral density of the noise projection of the object, and S Obj  is the average power spectral density of the noise-free projection of the object, resulting in a weighted image corresponding to said transformed captured image of the plurality of M transformed captured images, thereby obtaining a plurality of M weighted images;
 summing the weighted images of the plurality of M weighted images to obtain a summed transformed image; 
 inverse transforming the summed transformed image using inverse transform of the selected spatial transform to obtain a processed image; and 
 outputting the processed image, the step of outputting comprising storing the processed image, displaying the processed image, or transmitting the processed image; wherein: 
 one or more zeroes of optical transfer function of the optical system differ for at least two configurations of the plurality of different configurations corresponding to at least two images of the plurality of M captured images. 
 
     
     
         4 . The apparatus of  claim 3 , wherein:
 the selected spatial transform is a spatial Fourier Transform, and the inverse transform is an inverse spatial Fourier Transform; and   the configurable optical component comprises a deformable mirror configurable in the plurality of different configurations using at least one control parameter of the deformable mirror.   
     
     
         5 . An imaging method, the method comprising steps of:
 capturing a plurality of M captured images of an object in a scene through an optical system comprising an image sensor, the optical system being configurable in a plurality of configurations, each configuration of the plurality of configurations corresponding to a different optical transfer function between the object and the image sensor of the optical system, wherein each captured image of the plurality of M captured images is captured with a distinct optical transfer function between the object and the image sensor;   applying extended Wiener filtering to each captured image of the plurality of M captured images to obtain a corresponding filtered image, thereby obtaining a plurality of M filtered images;   combining computationally the plurality of M filtered images to obtain an enhanced image; and   at least one of displaying the enhanced image, storing the enhanced image in a machine-readable memory, and electronically transmitting the enhanced image.   
     
     
         6 . The imaging method according to  claim 5 , further comprising configuring the optical system in the distinct configurations of the plurality of configurations. 
     
     
         7 . The imaging method according to  claim 6 , wherein the step of configuring the optical system comprises causing the optical system to wait for the object to move to a different field of the optical system. 
     
     
         8 . The imaging method according to  claim 6 , wherein the step of configuring the optical system comprises changing optical axis of the optical system. 
     
     
         9 . The imaging method according to  claim 6 , wherein the step of configuring the optical system comprises providing relative motion between at least a portion of the optical system and the object. 
     
     
         10 . The imaging method according to  claim 6 , wherein the step of configuring the optical system comprises changing at least one control parameter of a deformable mirror of the optical system. 
     
     
         11 . The imaging method according to  claim 6 , wherein the step of configuring the optical system comprises moving a focal plane array of the optical system. 
     
     
         12 . The imaging method according to  claim 6 , wherein the step of configuring the optical system comprises altering focus of the optical system. 
     
     
         13 . The imaging method according to  claim 6 , wherein the step of configuring the optical system comprises changing dioptric power of a lens of the optical system. 
     
     
         14 . The imaging method according to  claim 6 , wherein the step of configuring the optical system comprises changing size of an optical aperture of the optical system. 
     
     
         15 . The imaging method according to  claim 6 , wherein the step of configuring the optical system comprises changing zoom setting of the optical system. 
     
     
         16 . The imaging method according to  claim 6 , wherein the step of configuring the optical system comprises panning the optical system. 
     
     
         17 . The imaging method according to  claim 6 , wherein the step of applying extended Wiener filtering comprises performing field correction of the captured images. 
     
     
         18 . The imaging method according to  claim 6 , wherein the step of applying extended Wiener filtering comprises performing spectral band correction of the captured images. 
     
     
         19 . An imaging apparatus, the apparatus comprising:
 an optical system comprising an image sensor, the optical system being configurable in a plurality of configurations, each configuration of the plurality of configurations corresponding to a different optical transfer function between an object in a scene and the image sensor of the optical system; and   at least one processor, wherein the at least one processor is coupled to the optical system to enable the at least one processor to control configuration of the optical system and to capture images in a focal plane of the optical system, and wherein the at least one processor is configured to execute program code instructions to cause the apparatus to:   capture a plurality of M captured images of the object, wherein each captured image of the plurality of M captured images is captured with a distinct optical transfer function between the object and the image sensor, at least one zero of the optical transfer function between the object and the image sensor being varied from image to image,   apply extended Wiener filtering to each captured image of the plurality of M captured images to obtain a corresponding filtered image, thereby obtaining a plurality of M filtered images, and   combine computationally the plurality of M weighted images to obtain an enhanced image; and   output the enhanced image by at least one of (1) displaying the enhanced image, (2) storing the enhanced image, and (3) transmitting the enhanced image.   
     
     
         20 . The imaging apparatus according to  claim 19 , wherein the optical system comprises a variable aperture, and the at least one processor is further configured to execute program code instructions to change size of the variable aperture of the optical system from image to image. 
     
     
         21 . The imaging apparatus according to  claim 19 , wherein the at least one processor is further configured to execute program code instructions to perform field-based correction of the captured images in the course of applying extended Wiener filtering. 
     
     
         22 . The imaging apparatus according to  claim 19 , wherein the at least one processor is further configured to execute program code instructions to perform spectral band correction of the captured images in the course of applying extended Wiener filtering. 
     
     
         23 . The imaging apparatus according to  claim 19 , wherein the optical system further comprises at least one of (1) a configurable liquid-crystal based spatial light modulator, and (2) a configurable liquid lens. 
     
     
         24 . The imaging apparatus according to  claim 19 , wherein the image sensor comprises a movable detector array.