Fast foveation camera and controlling algorithms

Abstract

Provided is a camera system to obtain high resolution imagery on multiple regions of interest even when the regions of interest are ate different focal depths and in different viewing directions. Embodiments include a high speed camera that operates by reflecting a beam of interest corresponding to a scene of interest into a high-speed passive sensor from a dynamic optical modulator. Embodiments described herein provide a foveating camera design that distributes resolution onto regions of interest by imaging reflections off a scanning micro-electromechanical system (MEMS) mirror. MEMS mirrors are used herein to modulate viewing direction. Embodiments include a camera capturing reflections off of a tiny, fast moving mirror.

The invention claimed is: 
     
       1. A camera comprising:
 a passive sensor; 
 a dynamic optical modulator; 
 an optical modulator controller configured to control an orientation of the dynamic optical modulator; and 
 a processor configured to receive digital image data from the passive sensor and to control the optical modulator controller, 
 wherein the dynamic optical modulator is configured to reflect a beam of light from a scene of interest onto the passive sensor, 
 wherein the processor is configured to control the optical modulator controller in accordance with a control algorithm, and 
 wherein the control algorithm provides a mirror speed for movement of the dynamic optical modulator and a frame-rate speed for capturing digital image data by the passive sensor, wherein the mirror speed and the frame-rate speed are calibrated by: 
 (a) an image is captured of a calibration scene using the camera; 
 (b) analysis of the image is performed to determine if a blur is present in the image; 
 (c) (i) reduction of a frame rate of the camera and returning to step (a) in response to a determination that no blur is present in the image;
 (ii) reduce a mirror speed of the dynamic optical modulator and returning to step (a) in response to a determination that a blur above a predetermined size is present in the image; and 
 (iii) determine that the control algorithm is calibrated in response to a determination that a blur below the predetermined size is present in the image. 
 
 
     
     
       2. The camera of  claim 1 , wherein the passive sensor is a CCD or CMOS image sensor. 
     
     
       3. The camera of  claim 1 , wherein the dynamic optical modulator is a scanning mirror or an array of scanning mirrors. 
     
     
       4. The camera of  claim 1 , wherein the dynamic optical modulator is a MEMs mirror or an array of MEMs mirrors. 
     
     
       5. The camera of  claim 4 , wherein each MEMs mirror comprises a plurality of microactuators. 
     
     
       6. The camera of  claim 5 , wherein a central reflective mirror plate of each MEMs mirror is suspended by four microactuators. 
     
     
       7. The camera of  claim 6 , wherein each of the microactuators are configured to be electrically controlled to set a height of a corresponding edge of the central reflective mirror plate, the microactuator being anchored to the corresponding edge. 
     
     
       8. The camera of  claim 5 , wherein each of the microactuators is made of a folded thin-film beam. 
     
     
       9. The camera of  claim 5 , wherein the optical modulator controller is configured to control each of the microactuators. 
     
     
       10. The camera of  claim 1 , further comprising a cover glass positioned between the dynamic optical modulator and the passive sensor, wherein the passive sensor and the dynamic optical modulator are positioned at angles relative to the cover glass established to dampen reflections from the cover glass. 
     
     
       11. A composite camera comprising:
 a secondary camera, 
 a primary camera, wherein the primary camera is the camera of  claim 1 . 
 
     
     
       12. The composite camera of  claim 11 , wherein the secondary camera is one of an optical camera, a thermal camera, or an ultra-violet camera. 
     
     
       13. The composite camera of  claim 11 , wherein a field of view of the secondary camera and a field of view of the primary camera overlap at least in part. 
     
     
       14. The composite camera of  claim 11 , wherein the secondary camera is configured to capture a first image of a scene and the primary camera is configured to capture one or more second images, each of the second images being of an area of interest within the scene. 
     
     
       15. The composite camera of  claim 14 , wherein each second image has a higher resolution than the first image. 
     
     
       16. The composite camera of  claim 14 , configured to associate with one or more second images with the first image and provide and/or store the first and second images. 
     
     
       17. The composite camera of  claim 14 , wherein a field of view of the secondary camera and a field of view of the primary camera only partially overlap. 
     
     
       18. A method of imaging a scene with a composite camera, the composite camera being the composite camera of  claim 11 , the method comprising:
 capturing a first image of a scene with the secondary camera; 
 analyzing the first image with a processor to identify one or more areas of interest within the scene; 
 causing the optical modulator to control the dynamic optical modulator such that the primary camera captures one or more second images, each second image being an image of one of the one or more areas of interest within the scene; 
 associating the first and second images; and 
 storing and/or providing the first and second images. 
 
     
     
       19. The method of  claim 18 , wherein each second image has a higher resolution than the associated first image. 
     
     
       20. The method of  claim 18 , further comprising performing post-processing on at least one second image to remove imaging artifacts from the at least one second image. 
     
     
       21. A method for calibrating a control algorithm for a mirror speed relative to a frame rate, the method comprising:
 (a) capturing an image of a calibration scene using a camera; 
 (b) analyzing the calibration scene to determine if a blur is present in the image; 
 (c) (i) responsive to determining that no blur is present in the image, reducing a frame rate of the camera and returning to step (a);
 (ii) responsive to determining that a blur above a predetermined size is present in the image, reducing a mirror speed of a dynamic optical modulator and returning to step (a); and 
 (iii) responsive to determining that a blur below the predetermined size is present in the image, determining that the control algorithm is calibrated. 
 
 
     
     
       22. The method of  claim 21 , wherein the calibration scene is a textured scene. 
     
     
       23. The method of  claim 22 , wherein the calibration scene is a texture plane. 
     
     
       24. The method of  claim 22 , wherein the textured scene is textured with at least one of a tactile texture or a visual texture.