Infrared imaging enhancement with fusion

Abstract

Techniques using small form factor infrared imaging modules are disclosed. An imaging system may include visible spectrum imaging modules, infrared imaging modules, and other modules to interface with a user and/or a monitoring system. Visible spectrum imaging modules and infrared imaging modules may be positioned in proximity to a scene that will be monitored while visible spectrum-only images of the scene are either not available or less desirable than infrared images of the scene. Imaging modules may be configured to capture images of the scene at different times. Image analytics and processing may be used to generate combined images with infrared imaging features and increased detail and contrast. Triple fusion processing, including selectable aspects of non-uniformity correction processing, true color processing, and high contrast processing, may be performed on the captured images. Control signals based on the combined images may be presented to a user and/or a monitoring system.

What is claimed is: 
     
       1. A system comprising:
 a memory adapted to receive a visible spectrum image of a scene from a visible spectrum imager and a plurality of infrared images of the scene from an infrared imager; and 
 a processor configured to communicate with the memory, wherein the processor is configured to:
 generate a blurred infrared image from at least one of the plurality of infrared images; 
 determine, for each row of the blurred infrared image, a corresponding row fixed pattern noise (FPN) correction term; 
 determine, for each column of the blurred infrared image, a corresponding column FPN correction term; 
 apply the row and column FPN correction terms to the blurred infrared image to provide a row and column FPN corrected blurred infrared image; 
 determine a plurality of non-uniformity correction (NUC) terms based, at least in part, on the row and column FPN corrected blurred infrared image; 
 apply the NUC terms to one of the plurality of the infrared images to remove noise from the one of the plurality of the infrared images to provide a corrected infrared image; 
 receive control parameters; 
 derive high spatial frequency content from at least one of the visible spectrum image and the corrected infrared image; and 
 generate a combined image comprising relative contributions of at least the high spatial frequency content, wherein the relative contributions are determined in real-time by a user adjusting the control parameters. 
 
 
     
     
       2. The system of  claim 1 , wherein the generating the blurred infrared image comprises:
 detecting motion of the infrared imager based on motion sensor data provided by a motion sensor; 
 receiving a set of the plurality of infrared in response to detected motion; 
 generating a blurred infrared image from the set of the plurality of infrared images based on an average of the set of the plurality of infrared images. 
 
     
     
       3. The system of  claim 1 , wherein:
 the applying the NUC terms to the one of the plurality of infrared images comprises selectively applying the NUC terms to the one of the plurality of infrared images to selectively remove noise from the one of the plurality of infrared images; and 
 the selectively applying the NUC terms is determined by the control parameters. 
 
     
     
       4. The system of  claim 1 , further comprising the infrared imager, wherein the infrared imager is configured to:
 generate the blurred infrared image of the scene; and 
 derive the plurality of NUC terms. 
 
     
     
       5. The system of  claim 1 , wherein:
 one or more of the control parameters are determined by one or more threshold values associated with at least one of the visible spectrum image, the plurality of infrared images, and the corrected infrared image. 
 
     
     
       6. The system of  claim 1 , further comprising a user interface configured to communicate with the memory and the processor, wherein the processor is configured to:
 receive user input from the user interface; and 
 determine one or more of the control parameters from the user input. 
 
     
     
       7. The system of  claim 1 , further comprising deriving color characteristics of the scene from at least one of the visible spectrum image and the infrared image, wherein:
 the color characteristics of the scene are derived from a chrominance component of the visible spectrum image; 
 the relative contributions include a luminance component of the corrected infrared image; and 
 the relative contribution of the luminance component of the corrected infrared image substantially matches the relative contribution of the color characteristics, as controlled by the control parameters. 
 
     
     
       8. The system of  claim 7 , wherein:
 the luminance component of the corrected infrared image is blended with a luminance component of the visible spectrum image before being contributed to the combined image. 
 
     
     
       9. The system of  claim 1 , wherein:
 the high spatial frequency content is derived from a luminance component of the visible spectrum image; 
 the relative contributions include a chrominance component of the corrected infrared image; and 
 the relative contribution of the chrominance component of the corrected infrared image substantially matches the relative contribution of the high spatial frequency content, as controlled by the control parameters. 
 
     
     
       10. The system of  claim 9 , wherein:
 the high spatial frequency content is blended with a luminance component of the corrected infrared image before being contributed to the combined image. 
 
     
     
       11. The system of  claim 1 , further comprising the infrared imager, wherein:
 the infrared imager comprises a focal plane array (FPA) configured to capture the plurality of infrared images of the scene; and 
 the FPA comprises an array of microbolometers adapted to receive a bias voltage selected from a range of approximately 0.2 volts to approximately 0.7 volts. 
 
     
     
       12. A method comprising:
 receiving a visible spectrum image of a scene from a visible spectrum imager and a plurality of infrared images of the scene from an infrared imager; 
 generating a blurred infrared image from at least one of the plurality of infrared images; 
 determining, for each row of the blurred infrared image, a corresponding row fixed pattern noise (FPN) correction term; 
 determining, for each column of the blurred infrared image, a corresponding column FPN correction term; 
 applying the row and column FPN correction terms to the blurred infrared image to provide a row and column FPN corrected blurred infrared image; 
 determining a plurality of non-uniformity correction (NUC) terms based, at least in part, on the row and column FPN corrected blurred infrared image; 
 applying the NUC terms to one of the plurality of the infrared images to remove noise from the one of the plurality of the infrared images to provide a corrected infrared image; 
 receiving control parameters; 
 deriving color characteristics of the scene from at least one of the visible spectrum image and the corrected infrared image; 
 and 
 generating a combined image comprising relative contributions of at least the color characteristics, wherein the relative contributions are determined in real-time by a user adjusting the control parameters. 
 
     
     
       13. The method of  claim 12 , wherein the generating the blurred infrared image comprises:
 detecting motion of the infrared imager based on motion sensor data provided by a motion sensor; 
 receiving a set of the plurality of infrared images in response to detected motion; and 
 generating the blurred infrared image from the set of the plurality of infrared images based on an average of the set of the plurality of infrared images. 
 
     
     
       14. The method of  claim 12 , wherein the applying the NUC terms to the one of the plurality of infrared images comprises selectively applying the NUC terms to the one of the plurality of infrared images. 
     
     
       15. The method of  claim 12 , wherein the infrared imager is configured to:
 generate the blurred infrared image of the scene by intentionally blurring one of the plurality of infrared images of the scene or by determining a set of the plurality of infrared images corresponding to detected motion and generating the blurred infrared image from the set of the plurality of infrared images based on an average of the set of the plurality of infrared images; and 
 determine the plurality of NUC terms. 
 
     
     
       16. The method of  claim 12 , wherein:
 one or more of the control parameters are determined by one or more threshold value; associated with at least one of the visible spectrum image, the plurality of infrared images, and the corrected infrared image. 
 
     
     
       17. The method of  claim 12 , further comprising:
 receiving user input from a user interface, wherein one or more of the control parameters are determined by the user input. 
 
     
     
       18. The method of  claim 12 , further comprising:
 receiving information from a sensor, wherein one or more of the control parameters are determined by the information, wherein the information is associated with an amount of available ambient light. 
 
     
     
       19. The method of  claim 12 , wherein:
 the color characteristics of the scene are derived from a chrominance component of the visible spectrum image; 
 the relative contributions include a luminance component of the corrected infrared image; and 
 the relative contribution of the luminance component of the corrected infrared image substantially matches the relative contribution of the color characteristics, as controlled by the control parameters. 
 
     
     
       20. The method of  claim 19 , wherein:
 the luminance component of the corrected infrared image is blended with a luminance component of the visible spectrum image before being contributed to the combined image. 
 
     
     
       21. The method of  claim 12 , further comprising deriving high spatial frequency content from at least one of the visible spectrum image and the infrared image, wherein:
 the high spatial frequency content is derived from a luminance component of the visible spectrum image; 
 the relative contributions include a chrominance component of the corrected infrared image; and 
 the relative contribution of the chrominance component of the corrected infrared image substantially matches the relative contribution of the high spatial frequency content, as controlled by the control parameters. 
 
     
     
       22. The method of  claim 21 , wherein:
 the high spatial frequency content is blended with a luminance component of the corrected infrared image before being contributed to the combined image. 
 
     
     
       23. A non-transitory machine-readable medium comprising a plurality of machine-readable instructions which when executed by one or more processors of a system are adapted to cause the system to perform a method comprising:
 receiving a visible spectrum image of a scene from a visible spectrum imager and a plurality of infrared images of the scene from an infrared imager; 
 generating a blurred infrared image least one of the plurality of infrared images; 
 determining, for each row of the blurred infrared image, a corresponding row fixed pattern noise (FPN) correction term; 
 determining, for each column of the blurred infrared image, a corresponding column FPN correction term; 
 applying the row and column FPN correction terms to the blurred infrared image to provide a row and column FPN corrected blurred infrared image; 
 determining a plurality of non-uniformity correction (NUC) terms based, at least in part, on the row and column FPN corrected blurred infrared image; 
 applying the NUC terms to one of the plurality of the infrared images to remove noise from the one of the plurality of the infrared images to provide a corrected infrared image; 
 receiving control parameters; 
 deriving high spatial frequency content from at least one of the visible spectrum image and the corrected infrared image; and 
 generating a combined image comprising relative contributions of at least the high spatial frequency content, wherein the relative contributions are determined in real-time by a user adjusting the control parameters. 
 
     
     
       24. The system of  claim 2 , further comprising the visible spectrum imager and the infrared imager, wherein the infrared imager comprises the motion sensor.