Optimization of optical systems for improved light field capture and manipulation

Abstract

According to various embodiments of the present invention, the optical systems of light field capture devices are optimized so as to improve captured light field image data. Optimizing optical systems of light field capture devices can result in captured light field image data (both still and video) that is cheaper and/or easier to process. Optical systems can be optimized to yield improved quality or resolution when using cheaper processing approaches whose computational costs fit within various processing and/or resource constraints. As such, the optical systems of light field cameras can be optimized to reduce size and/or cost and/or increase the quality of such optical systems.

What is claimed is: 
     
         1 . In a light field image capture device, the light field image capture device including at least one sensor and processing circuitry, the processing circuitry configured to process captured light field image data to generate visual output, a method for modifying pixel properties of the at least one sensor, the method comprising:
 capturing light field image data at the at least one sensor, the light field image data representing output of a microlens array, the light field image data including properties for each of a plurality of pixels of the at least one sensor;   at the processing circuitry, for each of at least one pixel in the at least one sensor, and based on light coordinates of the at least one pixel, modifying the pixel by adjusting at least one property of the pixel to compensate for non-uniform sampling due to other characteristics of the light field image capture device; and   sending the captured light field image data, including the modified at least one pixel, to the processing circuitry.   
     
     
         2 . The method of  claim 1 , wherein modifying the pixel by adjusting at least one property of the pixel to compensate for non-uniform sampling due to other characteristics of the light field image capture device comprises adjusting at least one property of the pixel to compensate for factors related to at least one of: image sensor angular response, a microlens array, and a main lens. 
     
     
         3 . The method of  claim 1 , wherein modifying the pixel by adjusting at least one property of the pixel to compensate for non-uniform sampling due to other characteristics of the light field image capture device comprises adjusting at least one of: exposure duration, gain, and bias for the pixel. 
     
     
         4 . The method of  claim 1 , wherein modifying the pixel by adjusting at least one property of the pixel to compensate for non-uniform sampling due to other characteristics of the light field image capture device comprises implementing pixel adjustment using at least one of: post-fabrication location trimming of a semiconductor, a digital or analog signal, and fusing a scale or offset into a semiconductor. 
     
     
         5 . A non-transitory computer program product for use in a light field image capture device, the light field image capture device including at least one sensor and processing circuitry, the processing circuitry configured to process captured light field image data to generate visual output, the non-transitory computer program product comprising instructions stored thereon, that when executed on the processing circuitry of the light field image capture device device, perform the steps of:
 causing the sensor to capture light field image data at the at least one sensor, the light field image data representing output of a microlens array, the light field image data including properties for each of a plurality of pixels of the at least one sensor;   for each of at least one pixel in the at least one sensor, and based on light coordinates of the at least one pixel, modifying the pixel by adjusting at least one property of the pixel to compensate for non-uniform sampling due to other characteristics of the light field image capture device; and   receiving the captured light field image data, including the modified at least one pixel, at the processing circuitry.   
     
     
         6 . The non-transitory computer program product of  claim 5 , wherein modifying the pixel comprises adjusting at least one property of the pixel to compensate for factors related to at least one of: image sensor angular response, a microlens array, and a main lens. 
     
     
         7 . The non-transitory computer program product of  claim 5 , wherein modifying the pixel comprises adjusting at least one of: exposure duration, gain, and bias for the pixel. 
     
     
         8 . A light field image capture device, comprising:
 a microlens array;   at least one sensor, configured to capture light field image data representing output of the microlens array, the light field image data including properties for each of a plurality of pixels of the at least one sensor;   a pre-processor, configured to, for each of at least one pixel in the at least one sensor, and based on light coordinates of the at least one pixel, modify the pixel by adjusting at least one property of the pixel to compensate for non-uniform sampling due to other characteristics of the light field image capture device; and   processing circuitry, configured to process the pre-processed light field image data to generate visual output.   
     
     
         19 . A light field image capture device, comprising:
 an aperture;   a main lens;   a microlens array having lenses arranged in a regular pattern;   at least one sensor having pixels arranged in a regular pattern, the at least one sensor configured to capture light field image data representing output of the microlens array, the microlens array and the at least one sensor interoperating to capture light field image data from light passing through the aperture and the main lens:   a pre-processor, configured to access light that has passed through the aperture and the main leans and to introduce random variations into at least one of: the regular pattern of lenses in the microlens array and the regular pattern of pixels in the at least one sensor;   processing circuitry, configured to, subsequent to introduction of the random variations, capture light field image data from the accessed light and process the pre-processed light field image data to generate visual output.