System and method for image analysis

Abstract

A method for image analysis, including recording an image sequence at a vehicle system mounted to a vehicle; automatically detecting an object within the image sequence with a detection module; automatically defining a bounding box about the detected object within each image of the image sequence; modifying the image sequence with the bounding boxes for the detected object to generate a modified image sequence; at a verification module associated with the detection module, labeling the modified image sequence as comprising one of a false positive, a false negative, a true positive, and a true negative detected object based on the bounding box within at least one image of the modified image sequence; training the detection module with the label for the modified image sequence; and automatically detecting objects within a second image sequence recorded with the vehicle system with the trained detection module.

We claim: 
     
         1 . A method for image analysis, comprising:
 detecting expert driver behavior at a motion sensor of a vehicle system mounted to a vehicle;   in response to detecting expert driver behavior, recording an image at the vehicle system;   automatically detecting an object within the image with a first module;   automatically defining a first bounding box about the detected object;   modifying the image with the first bounding box for the detected object to generate a first modified image;   at a first verification module associated with the first module, labeling the first bounding box within the modified image with a first label as one of a false positive, a false negative, a true positive, and a true negative detected object;   training the first module with the first label for the first bounding box; and   automatically detecting objects within a second image, recorded at the vehicle system, with the trained first module.   
     
     
         2 . The method of  claim 1 , further comprising automatically labeling the first bounding box with a second label corresponding to the expert driving behavior, and training the first module with the first label and the second label. 
     
     
         3 . The method of  claim 1 , wherein the motion sensor comprises an accelerometer that outputs an accelerometer signal indicative of vehicle motion, and wherein detecting expert driver behavior comprises recording the accelerometer signal exceeding a threshold amplitude. 
     
     
         4 . The method of  claim 1 , wherein the first module is executed by the vehicle system, the method further comprising transmitting the first modified image from the vehicle system to a remote computing system, remote from the vehicle, based on the object detected within the image. 
     
     
         5 . The method of  claim 1 , further comprising:
 automatically assigning an object class to the detected object with a second module to generate a classified object;   modifying the image with a second bounding box for the classified object to generate a second modified image, wherein the second modified image comprises a set of pixels;   at a second verification module associated with the object class, labeling the second bounding box within the second modified image with a second label as one of a false positive, a false negative, a true positive, and a true negative for the object class;   automatically labeling the second bounding box with a third label corresponding to the expert driving behavior; and   training the second module with the second label and the third label for the second bounding box within the second modified image;   
     
     
         6 . The method of  claim 5 , wherein the object class is one of a plurality of object classes, and wherein the second verification module is one of a plurality of verification modules, each associated with a different object class of the plurality of object classes. 
     
     
         7 . The method of  claim 1 , wherein the first and second module are a first and second level of a cascaded classification system. 
     
     
         8 . The method of  claim 1 , wherein the second modified image defines a set of pixels, and wherein labeling the second bounding box comprises:
 automatically determining a horizon line in the second modified image based on image fiducials, and   automatically labeling the detected object based on a relative location in the second modified image between the horizon line and the detected object.   
     
     
         9 . The method of  claim 1 , further comprising:
 determining a score for the driver based on historic driving sessions associated with the driver;   labeling the bounding box with a second label based on the score; and   training the first module with the first label and second label for the bounding box when the driver has score above a predetermined threshold.   
     
     
         10 . The method of  claim 1 , wherein the first and second verification modules each comprise a user interface, wherein labeling comprises:
 receiving a user selection of the bounding box within the modified image, wherein receiving the user selection automatically labels the bounding box as a false positive.   
     
     
         11 . The method of  claim 1 , further comprising receiving a user boundary input about an image region, generating a user bounding box about the image region based on the user boundary input, and wherein receiving the user boundary input automatically labels the user bounding box as a false negative. 
     
     
         12 . The method of  claim 1 , further comprising:
 recording a sequential image, wherein the image is a first frame of an image sequence recorded with the vehicle system, and the sequential image is a second frame of the image sequence;   automatically defining a tracked bounding box about the detected object within the second frame based upon a predicted trajectory of the detected object between the first and second frame; wherein the predicted trajectory is determined by a tracking module;   modifying the second frame with the tracked bounding box to generated a modified second frame;   at the verification module, labeling the tracked bounding box within the modified second frame as one of a false positive, a false negative, a true positive, and a true negative based on a comparison between the predicted trajectory and an actual trajectory of the detected object between the first and second frame;   training the tracking module with the label for the tracked bounding box; and   automatically tracking detected objects within a second image sequence recorded with the vehicle system based on the trained tracking module.   
     
     
         13 . A method for image analysis, comprising:
 recording an image sequence at a vehicle system mounted to a vehicle;   automatically detecting an object within the image sequence with a detection module;   automatically defining a bounding box about the detected object within each image of the image sequence;   modifying the image sequence with the bounding boxes for the detected object to generate a modified image sequence;   at a verification module associated with the detection module, labeling the modified image sequence with a first label comprising one of a false positive, a false negative, a true positive, and a true negative detected object based on the bounding box within at least one image of the modified image sequence;   determining driver behavior of a driver operating the vehicle using a motion sensor of the vehicle system;   labeling the modified image sequence with a second label comprising the driver behavior;   training the detection module with the first label and the second label for the modified image sequence; and   automatically detecting objects within a second image sequence recorded with the vehicle system with the trained detection module.   
     
     
         14 . The method of  claim 13 , further comprising:
 automatically assigning an object class to the detected object with a classification module to generate a classified object;   modifying the image sequence with a second bounding box for the classified object within each image of the image sequence to generate a second modified image sequence;   at a second verification module associated with the object class, labeling the second modified image sequence with a third label comprising one of a false positive, a false negative, a true positive, and a true negative for the object class based on the second bounding box within at least one image of the second modified image sequence; and   training the classification module with the third label for the second bounding box within the second modified image sequence and the second label; and   automatically classifying objects within the second image sequence recorded with the vehicle system with the trained classification module.   
     
     
         15 . The method of  claim 14 , wherein the object class is determined based on a roadway type on which the vehicle is driving, and wherein the roadway type on which the vehicle is driving is determined by a vibration sensor of the vehicle system. 
     
     
         16 . The method of  claim 13 , further comprising determining a score for the driver, based on historic driving sessions associated with the driver. 
     
     
         17 . The method of  claim 16 , wherein the detection module is trained with the first label and second label for the modified image sequence when the driver has score above a predetermined threshold, wherein the second label is determined based on the score. 
     
     
         18 . The method of  claim 13 , wherein the motion sensor comprises an accelerometer that outputs an accelerometer signal indicative of vehicle motion, and wherein determining driver behavior comprises recording the accelerometer signal exceeding a threshold amplitude 
     
     
         19 . The method of  claim 18 , wherein recording the image sequence is performed in response to the accelerometer signal exceeding the threshold amplitude. 
     
     
         20 . The method of  claim 13 , wherein automatically defining the bounding box about the detected object comprises defining a bounding box within a first image of the image sequence, and further comprising defining bounding boxes around the object within subsequent images of the image sequence based on a predicted trajectory of the object, wherein the predicted trajectory is based on the object class.