Imaging systems and methods for immersive surveillance

Abstract

Security guards at large facilities, such as airports, monitor multiple screens that display images from individual surveillance cameras dispersed throughout the facility. If a guard zooms with a particular camera, he may lose image resolution, along with perspective on the surrounding area. Embodiments of the inventive Imaging System for Immersive Surveillance (ISIS) solve these problems by combining multiple cameras in one device. When properly mounted, example ISIS systems offer up to 360-degree, 240-megapixel views on a single screen. (Other fields of view and resolutions are also possible.) Image-stitching software merges multiple video feeds into one scene. The system also allows operators to tag and follow targets, and can monitor restricted areas and sound an alert when intruders breach them.

What is claimed is: 
     
         1 . A method of determining a model representing views of a scene from cameras in an array of cameras, each camera in the array of cameras having a field of view that overlaps with the field of view of another camera in the array of cameras, the method comprising:
 (A) for each pair of overlapping fields of view:
 (A1) selecting image features in a region common to the overlapping fields of view; 
 (A2) matching points corresponding to a subset of the image features in one field of view in the pair of overlapping fields of view to points corresponding the subset of the image features in the other field of view in the pair of overlapping fields of view to form a set of matched points; 
   (B) merging at least a subset of each set of the matched points to form a set of merged points; and   (C) estimating parameters associated with each field of view based on the set of merged points to form the model representing the views of the scene.   
     
     
         2 . The method of  claim 1  wherein (A) comprises identifying candidate image features using a random sample and consensus procedure. 
     
     
         3 . The method of  claim 1  wherein (A2) comprises calculating a homography that maps points from one field of view in the pair of overlapping fields of view to the other field of view in the pair of overlapping fields of view. 
     
     
         4 . The method of  claim 1  wherein (A2) comprises:
 projecting points corresponding to the image features from one field of view in the pair of overlapping fields of view onto the other field of view in the pair of overlapping fields of view; and 
 for each projected point, determining whether the projected point lies within a predefined distance of a corresponding point in the other field of view. 
 
     
     
         5 . The method of  claim 4  further comprising:
 selecting the subset of each set of matched points from among those projected points that lie within the predefined distance of the corresponding points. 
 
     
     
         6 . The method of  claim 1  wherein (A) further comprises:
 (A3) repeating steps (A1) and (A2) for different image features in the region common to the overlapping fields of view. 
 
     
     
         7 . The method of  claim 1  further comprising:
 (D) calculating a re-projection error associated with model representing the views of the scene; and 
 (E) weighting the parameters to reduce the re-projection error. 
 
     
     
         8 . A method of compensating for imbalances in color and white levels in color images of respective portions of a wide-area scene, each color image comprising red, green, and blue color channels acquired by a respective camera in a camera array disposed to image the wide-area scene, the method comprising:
 (A) normalizing values representing the red, green, and blue color channels to a reference value representing a response of the cameras in the camera array to white light;   (B) equalizing the values representing the red, green, and blue color channels to red, green, and blue equalization values, respectively;   (C) identifying high- and low-percentile values among each of the red, green, and blue color channels; and   (D) scaling each of the red, green, and blue color channels based on the high- and low-percentile values to provide compensated values representing the red, green, and blue color channels.   
     
     
         9 . The method of  claim 8  wherein (B) further comprises:
 (B1) calculating red, green, and blue median values for the red, green, and blue color channels, respectively; 
 (B2) calculating red, green, and blue equalization coefficients by dividing the red, green, and blue median values, respectively, by the mean of the red, green, and blue median values; and 
 (B3) scaling the values representing the red, green, and blue color channels by the red, green, and blue equalization coefficients, respectively. 
 
     
     
         10 . The method of  claim 8 , wherein (A) further comprises:
 (A1) disposing the camera array inside a calibration hemisphere, the calibration hemisphere comprising:
 a diffusive hemisphere, 
 a reflective hemisphere disposed outside the diffusive hemisphere, and 
 a plurality of light emtting diodes (LEDs), disposed between the diffusive hemisphere and the reflective hemisphere, to emit light that is reflected by the reflective hemisphere and transmitted by the diffusive hemisphere; 
   (A2) capturing a first image of the plurality of LEDs using the camera array when the plurality of LEDs is off;   (A3) capturing a second image of the plurality of LEDs using the camera array when the plurality of LEDs is on; and   (A4) generating the reference value based at least in part on the first image captured in (A2) and the second image captured in (A3).