System and method for video and image compression

Abstract

Systems and methods for interest (ROI), or Frame Segmentation can be provided within a video stream, in real-time, or within a few milliseconds of video frame duration of 30 msec, or even in the sub-millisecond range. This video frame segmentation is the basis of Pre-ATR-based Ultra-Real-Time (PATURT) video compression. Additionally, morphing compression, and watermarking can be based on the PATURT. Example applications of the PATURT include ROI-based real-time video recording in “black-box” devices, recording aircraft accidents, or catastrophes.

We claim: 
     
         1 . A flight recording device, comprising:
 a housing;   at least one storage device disposed in the housing and comprising storage locations configured to store video information generated in an aircraft;   shock-absorbing material disposed between the solid state memory device and the housing;   a processing device coupled to the storage device and a memory coupled to the processor, the memory storing program instruction which, when executed, cause the processor to perform video image processing to compress the video information; the compression comprising:   a) selecting principal and secondary image frame signatures;   b) applying the selected signatures to select and extracting ROI contours; and   c) applying predictive morphing compression to the regions of interest (ROIs).   
     
     
         2 . The flight recording device according to  claim 1 , wherein predictive morphing compression comprises;
 formatting video frames of the video information within a skeleton based on a scaled-affine transform; and   forward correcting objects of the formatted video frames based on precoded structures of those objects.   
     
     
         3 . The flight recording device according to  claim 2 , wherein the object is an ROI of the frame. 
     
     
         4 . The flight recording device according to  claim 2 , wherein forward correcting objects of the formatted video frames is further based on knowledge of a stream of video frames at least one frame in advance. 
     
     
         5 . The flight recording device according to  claim 2 , wherein the scaled affine transform comprises an instruction to rescale a coordinate of an edge of the object by a specified scale factor. 
     
     
         6 . The flight recording device according to  claim 5 , wherein the object is an ROI and the instruction comprises
   IF(ROI)=(ROI1),AND  x=x   1 ,THEN RE-SCALE  Y -COORDINATE BY ( a′/a )−FACTOR EQUAL TO  X  %.
   
     
     
         7 . The flight recording device according to  claim 5 , wherein the instruction comprises a scaling instruction based on fuzzy logic. 
     
     
         8 . The flight recording device according to  claim 5 , further comprising transmitting the video information, wherein transmission of the video information comprises sending a first frame of video information and, instead of sending a subsequent second frame, sending the scaled affine transform instruction in place of the second frame. 
     
     
         9 . The flight recording device according to  claim 5 , wherein the compression using predictive morphing compression achieves a compression ratio of up to 100,000:1. 
     
     
         10 . The flight recording device according to  claim 1 , wherein the shock absorbing material in combination with the device provides an impact acceleration having a magnitude of at least 3,400 G's. 
     
     
         11 . The flight recording device according to  claim 1 , wherein the shock absorbing material in combination with the device provides an impact acceleration having a magnitude of at least 4,000 G's. 
     
     
         12 . The flight recording device according to  claim 1 , wherein the housing comprises titanium. 
     
     
         13 . The flight recording device according to  claim 1 , wherein the storage device is a solid state memory device disposed outside the housing. 
     
     
         14 . The flight recording device according to  claim 1 , further comprising a removable printed circuit board and a mating connector fixedly attached to the removable printed circuit board, wherein reconfiguration of the flight recording device for a particular aircraft is effectuated by replacement of the connector is effected by replacement of said removable printed circuit board. 
     
     
         15 . The flight recording device according to  claim 14 , wherein the a removable printed circuit board comprises:
 a second connector that is removably matable with a corresponding connector on a main board of the flight recording device;   matching circuitry on the able printed circuit board connected to match pins on the mating connector with pins on the second connector thereby allowing a universal flight recording device to be integrated with a particular aircraft.   
     
     
         16 . The flight recording device according to  claim 1 , wherein said memory device is configured for recording up to four channels of audio information and up to two channels of video information. 
     
     
         17 . The flight recording device according to  claim 1 , wherein said memory device is configured for recording up to eight channels of discrete data and up to sixteen channels of analog data. 
     
     
         18 . The flight recording device according to  claim 1 , wherein the housing comprises a capsule housing the storage device, the capsule being filled the shock absorbing material and the shock absorbing material surrounding the memory device. 
     
     
         19 . The flight recording device according to  claim 1 , wherein the shock absorbing material comprises Aerogel. 
     
     
         20 . The flight recording device according to  claim 1 , further comprising:
 identifying sensor values representing the video image information as matrix elements in a multidimensional matrix form;   treating a set of matrix elements as a gray-scale image and associating each matrix element with a corresponding pixel value relative to the gray-scale image; and wherein at least steps b) and c) are applied to the image based on the associated pixel values.   
     
     
         21 . The flight recording device according to  claim 1 , wherein the aircraft is an unmanned aerial vehicle. 
     
     
         22 . A flight recording device, comprising:
 a housing;   at least one storage device disposed in the housing and comprising storage locations configured to store video information generated in an aircraft;   shock-absorbing material disposed between the solid state memory device and the housing; and   a processing device coupled to the storage device and a memory coupled to the processor, the memory storing program instruction which, when executed, cause the processor to perform video image processing to compress the video information; the compression comprising:   a) sequentially selecting a plurality of N principal signatures based on time available for computation and bandwidth available for data transfer;   b) applying the selected signatures to select and extracting ROI contour boundaries to divide the frame area into a plurality of ROIs and a remaining background; and   c) applying multifaceted inhomogeneous compression to the frame.   
     
     
         23 . The flight recording device according to  claim 22 , wherein multifaceted inhomogeneous compression comprises:
 determining whether the plurality of ROIs in the background can be compressed differently and with different compression ratios; and   applying a different compression ratio to at least one ROI as compared to the compression ratio applied to the other ROI's or the background.   
     
     
         24 . The flight recording device according to  claim 22 , wherein multifaceted inhomogeneous compression comprises selectively scrambling information in at least one of the ROIs. 
     
     
         25 . The flight recording device according to  claim 22 , wherein multifaceted inhomogeneous compression comprises applying different compression ratios to different ROIs. 
     
     
         26 . The flight recording device according to  claim 22 , wherein multifaceted inhomogeneous compression comprises applying different compression schemes to different ROIs. 
     
     
         27 . The flight recording device according to  claim 22 , wherein multifaceted inhomogeneous compression comprises identifying an ROI comprising a face and applying a sufficient level of compression to the ROI to render the face unrecognizable. 
     
     
         28 . The flight recording device according to  claim 22 , wherein sequentially selecting a plurality of N principal signatures, selecting and extracting an ROI contour boundary, and applying multifaceted inhomogeneous compression is performed in less than 30 milliseconds. 
     
     
         29 . The flight recording device according to  claim 22 , wherein sequentially selecting a plurality of N principal signatures, selecting and extracting an ROI contour boundary, and applying multifaceted inhomogeneous compression is performed in less than 1 millisecond. 
     
     
         30 . The flight recording device according to  claim 22 , wherein the selection of signatures is performed using false target rejection, characterized by a high false alarm rate and a low false negative rate. 
     
     
         31 . The flight recording device according to  claim 22 , wherein applying the signatures comprises:
 applying a first signature of the selected signatures to a region of the image using conditional probabilities to identify and reject false targets and arrive at a first target set, and   applying to the first target set a second signature that is more selective than the first signature to identify and reject false targets in the first target set.   
     
     
         32 . The flight recording device according to  claim 22 , wherein sequentially selecting a plurality of N principal signatures comprises pixel-wise intensity subtraction. 
     
     
         33 . The flight recording device according to  claim 32 , wherein pixel-wise intensity subtraction is performed between pixel intensities as pixel-to-pixel, frame-to-frame, object-to-object, frame-to-reference frame, object-to-reference object, intensity-to-threshold value, or ROI-to-ROI. 
     
     
         34 . The flight recording device according to  claim 22 , wherein sequentially selecting a plurality of N principal signatures comprises computing a distance between two pixel units. 
     
     
         35 . The flight recording device according to  claim 34 , wherein computing a distance between two pixel units comprises performing a pixel-by-pixel comparison between corresponding pixel units of different frames, or a pixel-by-pixel comparison between sequential pixel units of the same frame. 
     
     
         36 . The flight recording device according to  claim 34 , wherein a pixel unit comprises a pixel, a pixel cluster or a template. 
     
     
         37 . The flight recording device according to  claim 22 , wherein sequentially selecting a plurality of N principal signatures comprises speed vector flow mapping. 
     
     
         38 . The flight recording device according to  claim 22 , wherein sequentially selecting a plurality of N principal signatures comprises color matching. 
     
     
         39 . The flight recording device according to  claim 22 , wherein applying the selected signatures to select and extract ROI contour boundaries comprises filtering, decimation and pixel-by-pixel subtraction. 
     
     
         40 . The flight recording device according to  claim 22 , wherein the selecting and extracting an ROI contour boundary comprises phase-space-scaling (PSS) in a 4 dimensional space to scale the frame phase space domain into a smaller size, thereby reducing transmission bandwidth and shortening computation time. 
     
     
         41 . The flight recording device according to  claim 22 , wherein the selecting and extracting an ROI contour boundary comprises edge extraction based on phase space scaling. 
     
     
         42 . The flight recording device according to  claim 41 , wherein the edge extraction comprises image line decimation and smoothing. 
     
     
         43 . The flight recording device according to  claim 22 , wherein the selecting and extracting an ROI contour boundary comprises determining the center of gravity of each ROI using a polar contour compliance procedure. 
     
     
         44 . The flight recording device according to  claim 22 , further comprising a removable printed circuit board and a mating connector fixedly attached to the removable printed circuit board, wherein reconfiguration of the flight recording device for a particular aircraft is effectuated by replacement of the connector is effected by replacement of said removable printed circuit board. 
     
     
         45 . The flight recording device according to  claim 44 , wherein the a removable printed circuit board comprises:
 a second connector that is removably matable with a corresponding connector on a main board of the flight recording device;   matching circuitry on the able printed circuit board connected to match pins on the mating connector with pins on the second connector thereby allowing a universal flight recording device to be integrated with a particular aircraft.   
     
     
         46 . The flight recording device according to  claim 22 , further comprising:
 identifying sensor values representing the video image information as matrix elements in a multidimensional matrix form;   treating a set of matrix elements as a gray-scale image and associating each matrix element with a corresponding pixel value relative to the gray-scale image; and wherein at least steps b) and c) are applied to the image based on the associated pixel values.   
     
     
         47 . The flight recording device according to  claim 22 , wherein the aircraft is an unmanned aerial vehicle. 
     
     
         48 . A method for applying a digital watermarking to a plurality of video frames, comprising:
 a processor system identifying and extracting an ROI contour boundary to divide the frame area into one or more ROIs and remaining background; and   the processor system identifying low-importance bits in the background and altering one or more of the identified low-importance bits outside of the ROI and within inter-frame video streams.   
     
     
         49 . The method of  claim 48 , wherein altering one or more of the identified low-importance bits outside of the ROI comprises changing a least significant bit of a binary intensity number from a “0” to a “1” or from a “1” to a “0”. 
     
     
         50 . A method for applying a digital watermarking to a plurality of video frames, comprising:
 a processor system identifying and extracting an ROI contour boundary to divide the frame area into one or more ROIs and remaining background; and   alternating the video image without perceptual change, and adding hidden information into the background outside of the ROIs.   
     
     
         51 . A crash survivable video recorder for aircraft cockpit event recording of events occurring up to no more than 0.5 second prior to a catastrophic event, the video recorder comprising:
 a video camera for capturing frames of video data; and   image processing electronics coupled to receive the captured frames of video data and configured to analyze each frame of video data generated in said camera; determine various regions of interest in said frame; and selectively compressing data in each of the regions of interest by a compression ratio that is dependent upon the information content in each region of interest.   
     
     
         52 . The video recorder of  claim 51 , wherein any region of interest including a pilot's facial features is automatically compressed by a compression ratio sufficiently high to obscure the pilot's face. 
     
     
         53 . A method of intraframe real time, video image processing for reducing bandwidth and computation time; the method comprising the following steps: a) selecting image frame signatures; b) filtering and decimation; c) pixel-wise subtraction; and d) edge smoothing; wherein steps a), b), c) and d) are all performed within the duration of a single frame of video. 
     
     
         54 . The method recited in  claim 53 , wherein step b) is carried out using the steps of filtering, decimation and pixel subtraction. 
     
     
         55 . The method recited in  claim 53 , wherein the steps of filtering and decimation are carried out using a single step of filtering and decimation employing phase space filter banks. 
     
     
         56 . The method recited in  claim 53 , wherein the steps of filtering and decimation are carried out using a single step of filtering and decimation by a phase space scaling operation. 
     
     
         57 . A flight recording device, comprising:
 a housing;   at least one storage device disposed in the housing and comprising storage locations configured to store video information generated in an aircraft;   at least one sensor communicatively coupled to the storage device and configured to capture video image information; and   a processing device coupled to the storage device and a memory coupled to the processor, the memory storing program instruction which, when executed, cause the processor to perform video image processing to compress the video information; the compression comprising:   a) identifying sensor values representing the video image information as matrix elements in a multidimensional matrix form;   b) treating a set of matrix elements as a gray-scale image and associating each matrix element with a corresponding pixel value relative to the gray-scale image; and   c) applying object-oriented compression to the image based on the associated pixel values.   
     
     
         58 . The flight recording device according to  claim 57 , wherein the aircraft is an unmanned aerial vehicle. 
     
     
         59 . A flight recording device for recording events on an aircraft, comprising:
 a housing;   a printed circuit board disposed within the housing;   a processing device and a memory coupled to the processor, forming a part of a circuit and mounted to the printed circuit board;   a first connector mounted on the printed circuit board and electrically coupled to the circuit;   a connector interface board comprising:
 a second connector that is removably matable with the first connector; 
 a third connector mounted on the connector interface board and comprising a plurality of connector pins that are pin-for-pin compatible with a connector of the aircraft; 
   matching circuitry on the connector interface board connected to match pins on the third connector with pins on the second connector thereby allowing a universal flight recording device to be integrated with a particular aircraft.