Enhanced flight vision system

Abstract

An enhanced vision system comprising a camera, a memory, and a processor. The camera configured to capture video of a field of view and output a plurality of frames, each frame including a respective sensor pixel value for each of a plurality of sensor pixels. The memory configured to store the plurality of frames and a threshold sensor pixel value. The processor configured to receive the plurality of frames, identify, in each of the plurality of frames output over a first period of time, one or more sensor pixel values above a threshold sensor pixel value, enhance a current frame by changing corresponding sensor pixel values of the current frame based on each of the identified one or more sensor pixel values above the threshold sensor pixel value, and generate an enhanced video of the field of view including the enhanced current frame to be presented on a display.

What is claimed is: 
     
       1. An enhanced vision system operable to be used with an aircraft, the enhanced vision system comprising:
 a camera including a plurality of sensor pixels, the camera configured to capture video of a field of view and output a plurality of frames, each frame including a respective sensor pixel value for each of the plurality of sensor pixels; 
 a memory configured to store the plurality of frames and a threshold sensor pixel value; and 
 a processor coupled to the camera and the memory, the processor configured to:
 receive, from the memory, the plurality of frames output by the camera over a first period of time, the plurality of frames including a current frame; 
 identify, in each of the plurality of frames output over the first period of time, one or more sensor pixel values above the threshold sensor pixel value; 
 determine a highest sensor pixel value in the plurality of frames output over the first period of time for each sensor pixel value above the threshold sensor pixel value; 
 enhance the current frame by changing corresponding sensor pixel values of the current frame based on each of the identified one or more sensor pixel values above the threshold sensor pixel value, the sensor pixel values of the current frame changed to the highest determined sensor pixel value; and 
 generate an enhanced video of the field of view including the enhanced current frame to be presented on a display, the display having a plurality of display pixels corresponding to the sensor pixel values. 
 
 
     
     
       2. The enhanced vision system of  claim 1 , wherein the memory is further configured to store a predetermined sensor pixel value, and wherein the processor is further configured to receive the predetermined sensor pixel value and change each sensor pixel value above the threshold sensor pixel value to the predetermined sensor pixel value to enhance the current frame. 
     
     
       3. The enhanced vision system of  claim 1 , wherein the processor is further configured to:
 designate sensor pixel values determined to be above the threshold sensor pixel value as being in a high state; and 
 designate sensor pixel values determined to be below the threshold sensor pixel value as being in a low state. 
 
     
     
       4. The enhanced vision system of  claim 3 , wherein the processor is further configured to determine that a particular sensor pixel value changed from the low state to the high state and then back to the low state in three sequential frames, and in response the processor is configured to enhance the current frame by changing the corresponding sensor pixel value to a highest sensor pixel value of the particular sensor pixel value within the three sequential frames. 
     
     
       5. The enhanced vision system of  claim 3 , wherein the processor is further configured to determine that a particular sensor pixel value changed from the low state to the high state, remained in the high state in the next frame, and then back to the low state in four sequential frames, and in response the processor is configured to enhance the current frame by changing the corresponding sensor pixel value to a highest sensor pixel value of the particular sensor pixel value within the four sequential frames. 
     
     
       6. The enhanced vision system of  claim 3 , wherein the processor is further configured to determine that a particular sensor pixel value changed from the low state to the high state and then back to the low state during at least a portion of the first period of time. 
     
     
       7. The enhanced vision system of  claim 3 , wherein the processor is further configured to:
 determine an average sensor pixel value for each of the identified one or more sensor pixel values above the threshold sensor pixel value; and 
 change the corresponding sensor pixel values of the current frame to the determined average sensor pixel values for each of the one or more sensor pixel values above the threshold sensor pixel value to enhance the current frame. 
 
     
     
       8. The enhanced vision system of  claim 1 , wherein the camera comprises a CMOS sensor, and wherein the camera has a frame rate that is greater than a refresh rate of the display. 
     
     
       9. The enhanced vision system of  claim 1 , wherein the processor is further configured to determine weather conditions proximate to a landing location for aircraft and adjust the stored threshold sensor pixel value in response to determining weather conditions that limit visibility. 
     
     
       10. The enhanced vision system of  claim 1 , wherein the processor is further configured to generate a mask associated with sensor pixels of the current frame having sensor pixel values above the threshold sensor pixel value in at least one of the plurality of frames output over the first period of time, and wherein the processor changes each of the sensor pixel values above the threshold sensor pixel value to enhance the current frame by combining the current frame with the mask. 
     
     
       11. The enhanced vision system of  claim 1 , wherein a portion of the field of view comprises a flashing light of an aircraft approach lighting system proximate to a landing location for aircraft. 
     
     
       12. The enhanced vision system of  claim 1 , wherein a portion of the field of view comprises a light originating from another aircraft between the aircraft and a landing location for aircraft. 
     
     
       13. An enhanced vision system operable to be used with an aircraft, the enhanced vision system comprising:
 a camera including a plurality of sensor pixels, the camera configured to capture video of a field of view and output a plurality of frames, each frame including a respective sensor pixel value for each of the plurality of sensor pixels; 
 a display having a plurality of display pixels corresponding to the sensor pixel values; 
 a memory configured to store the plurality of frames, a threshold sensor pixel value and a predefined pattern; and 
 a processor coupled to the camera and the memory, the processor configured to:
 receive, from the memory, the plurality of frames output by the camera over a first period of time, the plurality of frames including a current frame; 
 identify, in each of the plurality of frames output over the first period of time, one or more sensor pixel values of a plurality of sensor pixel values above the threshold sensor pixel value; 
 detect the predefined pattern for a sensor pixel over the first period of time based on the sensor pixel values in a subset of the plurality of frames output by the camera; 
 enhance the current frame by changing corresponding sensor pixel values of the current frame to a highest sensor pixel value of the identified one or more sensor pixel values above the threshold sensor pixel value in response to the predefined pattern being detected; and 
 generate, and output to the display, an enhanced video of the field of view including the enhanced current frame to be presented on the display. 
 
 
     
     
       14. The enhanced vision system of  claim 13 , wherein the processor is further configured to detect the predefined pattern in response to a particular sensor pixel value changing from a low state to a high state, remaining in the high state in a next frame, and then going back to the low state in four sequential frames. 
     
     
       15. The enhanced vision system of  claim 13 , wherein the camera comprises a CMOS sensor, and herein the camera has a frame rate that is greater than a refresh rate of the display. 
     
     
       16. The enhanced vision system of  claim 13 , wherein a portion of the field of view comprises one of a flashing light of an aircraft approach lighting system proximate to a landing location for aircraft or a light originating from another aircraft between the aircraft and a landing location for aircraft. 
     
     
       17. An enhanced vision system operable to be used with an aircraft, the enhanced vision system comprising:
 a camera including a plurality of sensor pixels, the camera configured to capture video of a field of view and output a plurality of frames, each frame including a respective sensor pixel value for each of the plurality of sensor pixels; 
 a display having a plurality of display pixels corresponding to the sensor pixel values; 
 a memory configured to store the plurality of frames and a threshold sensor pixel value; and 
 a processor coupled to the camera, the display and the memory, the processor configured to:
 receive, from the memory, the plurality of frames output by the camera over a first period of time, the plurality of frames including a current frame; 
 identify, in each of the plurality of frames output over the first period of time, one or more sensor pixel values above the threshold sensor pixel value as a high state; 
 enhance the current frame by changing corresponding sensor pixel values of the current frame to an average of each of the one or more sensor pixel values identified as being in the high state; and 
 generate, and output to the display, an enhanced video of the field of view including the enhanced frame to be presented on the display; 
 
 wherein the processor is further configured to increase the threshold sensor pixel value in response to identifying a high level of ambient light and decrease the threshold sensor pixel value in response to identifying a low level of ambient light or weather conditions that limit visibility. 
 
     
     
       18. The enhanced vision system of  claim 17 , wherein the processor is further configured to set the threshold sensor pixel value based on a stored intensity of a flashing light of an aircraft approach lighting system, and wherein a portion of the field of view comprises a flashing light of an aircraft approach lighting system proximate to a landing location for aircraft.