Optimization of optical systems for improved light field capture and manipulation

Abstract

According to various embodiments of the present invention, the optical systems of light field capture devices are optimized so as to improve captured light field image data. Optimizing optical systems of light field capture devices can result in captured light field image data (both still and video) that is cheaper and/or easier to process. Optical systems can be optimized to yield improved quality or resolution when using cheaper processing approaches whose computational costs fit within various processing and/or resource constraints. As such, the optical systems of light field cameras can be optimized to reduce size and/or cost and/or increase the quality of such optical systems.

What is claimed is: 
     
         1 . In a light field image capture device, the light field image capture device including a microlens array, at least one sensor, a disk pattern formed on the sensor, a color filter array (CFA) grid layout, and processing circuitry, the processing circuitry configured to process captured light field image data to generate visual output, the disk pattern and the color filter array (CFA) interoperating to convert output from the microlens array into captured light field image data, a method for modifying the color filter array (CFA), the method comprising:
 accessing the configuration of the disk pattern;   at the processing circuitry, determining how to modify the color filter array (CFA) to improve the quality of the color information captured from the output of the microlens array;   modifying the color filter array (CFA) in accordance with the determination;   capturing light field image data from the output of the microlens array in accordance with the disk pattern and the modified color filter array (CFA); and   sending the captured light field image data to the processing circuitry.   
     
     
         2 . The method of  claim 1 , wherein modifying the color filter array (CFA) in accordance with the determination comprises modifying the color filter array so that no color filter is applied to at least one pixel. 
     
     
         3 . The method of  claim 1 , wherein modifying the color filter array (CFA) in accordance with the determination comprises modifying color filter pass bands for at least one pixel. 
     
     
         4 . The method of  claim 1 , wherein modifying the color filter array (CFA) in accordance with the determination comprises randomizing a color filter array (CFA) pattern. 
     
     
         5 . A method for determining a configuration for a microlens array that is to be positioned over an image sensor for use in a light field image capture device, the method comprising:
 accessing characteristics of the microlens array;   accessing characteristics of the image sensor, including the read-out modes of the image sensor;   at a processor, determining an optimal configuration for the microlens array based on the characteristics of the microlens array and the characteristics of the image sensor, including the read-out modes of the image sensor; and   storing, in a storage device, the optimal configuration for application to the microlens array during manufacture of the light field capture device.   
     
     
         6 . The method of  claim 5 , wherein determining an optimal configuration for the microlens array comprises determining an optimal position for the microlens array over the image sensor. 
     
     
         7 . The method of  claim 5 , wherein determining an optimal configuration for the microlens array comprises determining an optimal orientation for the microlens array over the image sensor. 
     
     
         8 . The method of  claim 5 , wherein determining an optimal configuration for the microlens array comprises considering at least one physical rotation of the microlens array about an axis orthogonal to the image sensor. 
     
     
         9 . The method of  claim 5 , wherein determining an optimal configuration for the microlens array comprises considering at least one physical translation of the microlens array in a plane parallel to the image sensor plane. 
     
     
         10 . The method of  claim 5 , wherein determining an optimal configuration for the microlens array comprises considering at least one read-out mode of the sensor 
     
     
         11 . The method of  claim 10 , wherein each read-out modes is selected from the group consisting of: binning, skipping, resampling, and scaling. 
     
     
         12 . A non-transitory computer program product for use in a light field image capture device, the light field image capture device including a microlens array, at least one sensor, a disk pattern formed on the sensor, a color filter array (CFA) grid layout, and processing circuitry, the processing circuitry configured to process captured light field image data to generate visual output, the disk pattern and the color filter array (CFA) interoperating to convert output from the microlens array into captured light field image data, the non-transitory computer program product comprising instructions stored thereon, that when executed on the processing circuitry of the light field image capture device device, perform the steps of:
 accessing the configuration of the disk pattern;   determining how to modify the color filter array (CFA) to improve the quality of the color information captured from the output of the microlens array;   modifying the color filter array (CFA) in accordance with the determination;   causing the sensor to capture light field image data from the output of the microlens array in accordance with the disk pattern and the modified color filter array (CFA); and   receiving the captured light field image data at the processing circuitry.   
     
     
         13 . A light field image capture device, comprising:
 a microlens array;   at least one sensor, configured to capture light field image data representing output of the microlens array, the light field image data comprising a disk pattern formed on the sensor, the at least one sensor having a color filter array (CFA) grid layout;   a pre-processor, configured to access the configuration of the disk pattern and to determine how to modify the color filter array (CFA) to improve the quality of the color information captured from the output of the microlens array;   a CFA modifier, configured to modify the color filter array (CFA) in accordance with the determination; and   processing circuitry, configured to process the pre-processed light field image data to generate visual output;   wherein the disk pattern and the color filter array (CFA) interoperate to convert output from the microlens array into captured light field image data;   and wherein the at least one sensor captures light field image data from the output of the microlens array in accordance with the disk pattern and the modified color filter array (CFA).