Gunshot detection using multichannel analysis

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for gunshot detection using multichannel analysis. One of the methods includes the actions of determining, by a device that includes an infrared sensor and an acoustic sensor, whether infrared data captured by the infrared sensor satisfies a saturation threshold; selecting at least one classifier using a result of the determination whether the infrared data satisfies the saturation threshold; accessing at least one feature from one or more of the infrared data captured by the infrared sensor or acoustic data captured by the acoustic sensor; determining whether the at least one feature represents a possible gunshot event in an environment that includes the device; and performing an action using a result of the determination whether the at least one feature represents a possible gunshot event in the environment.

What is claimed is: 
     
         1 . A computer-implemented method comprising:
 determining, by a device that includes an infrared sensor and an acoustic sensor, whether infrared data captured by the infrared sensor satisfies a saturation threshold;   selecting, from a plurality of classifiers, at least one classifier using a result of the determination whether the infrared data satisfies the saturation threshold;   accessing at least one feature from one or more of the infrared data captured by the infrared sensor or acoustic data captured by the acoustic sensor;   determining, using the at least one classifier and the at least one feature, whether the at least one feature represents a possible gunshot event in an environment that includes the device; and   performing an action using a result of the determination whether the at least one feature represents a possible gunshot event in the environment.   
     
     
         2 . The method of  claim 1 , wherein:
 the at least one feature comprises an infrared amplitude feature;   determining whether the infrared amplitude feature satisfies the saturation threshold comprises determining whether the infrared amplitude feature satisfies a peak amplitude threshold; and   selecting at least one classifier comprises, in response to determining that the infrared amplitude feature satisfies the peak amplitude threshold, selecting, from the plurality of classifiers that includes saturated classifiers and unsaturated classifiers, at least one saturated classifiers.   
     
     
         3 . The method of  claim 1 , wherein:
 the at least one feature comprises an infrared amplitude feature;   determining whether the infrared amplitude feature satisfies the saturation threshold comprises determining whether the infrared amplitude feature satisfies a peak amplitude threshold; and   selecting at least one classifier comprises, in response to determining that the infrared amplitude feature does not satisfy the peak amplitude threshold, selecting, from the plurality of classifiers that includes saturated classifiers and unsaturated classifiers, at least one unsaturated classifiers.   
     
     
         4 . The method of  claim 3 , wherein:
 the infrared amplitude feature comprises a sailboat feature; and   selecting at least one classifier comprises, in response to determining that the sailboat feature does not satisfy the peak amplitude threshold, selecting, from the plurality of classifiers that includes saturated classifiers and unsaturated classifiers, at least one unsaturated classifiers.   
     
     
         5 . The method of  claim 1 , wherein:
 accessing the at least one feature from the one or more of the infrared data captured by the infrared sensor or the acoustic data captured by the acoustic sensor comprises accessing at least an infrared peak feature from the infrared data and an acoustic peak feature from the acoustic data;   determining, using the at least one classifier and the at least one feature, whether the at least one feature represents the possible gunshot event in the environment that includes the device comprises:
 determining whether the infrared peak feature and the acoustic peak feature satisfy a peak timing threshold; and 
 determining, in response to determining that the infrared peak feature and the acoustic peak feature satisfy the peak timing threshold and using the at least one classifier and the at least one feature, whether the at least one feature represents the possible gunshot event in the environment that includes the device. 
   
     
     
         6 . The method of  claim 1 , wherein:
 determining whether the at least one feature represents a possible gunshot event in the environment comprises:
 providing the at least one feature to the at least one classifier; 
 receiving from at least one of the classifiers a detection score that indicates a likelihood of a possible gunshot event in the environment; 
 determining whether the detection score of the possible gunshot event in the environment satisfies a detection score threshold; and 
 in response to determining that the detection score of the possible gunshot event in the environment satisfies the detection score threshold, determining that the at least one feature represents a possible gunshot event in the environment; and 
   performing the action comprises, in response to determining that the at least one feature represents a possible gunshot event in the environment, providing at least some data from the infrared data or the acoustic data to at least one solver.   
     
     
         7 . The method of  claim 6 , comprising:
 determining, using the at least one solver, whether the possible gunshot event is likely an actual gunshot event in the environment; and   selectively triggering an event or determining to skip triggering an event using a result of the determination whether the possible gunshot event is likely an actual gunshot event in the environment.   
     
     
         8 . The method of  claim 7 , comprising:
 in response to determining that the possible gunshot event is likely an actual gunshot event in the environment, determining, using a weapon classifier, a predicted weapon type of the actual gunshot event; and   selectively triggering an event using the predicted weapon type.   
     
     
         9 . The method of  claim 8 , wherein determining the predicted weapon type of the actual gunshot event comprises determining, using the weapon classifier and an earth mover distance feature, the predicted weapon type of the actual gunshot event. 
     
     
         10 . The method of  claim 1 , wherein:
 the at least one classifier comprises at least two classifiers, and   determining whether the at least one feature represents a possible gunshot event in the environment comprises:
 receiving, from each of the at least two classifiers, a corresponding detection score that indicates a likelihood of a possible gunshot event in the environment, 
 generating an aggregated detection score by combining the corresponding detection scores from the at least two classifiers, 
 determining whether the aggregated detection score satisfies a detection score threshold, and 
 in response to determining that the aggregated detection score of the possible gunshot event in the environment satisfies the detection score threshold, determining that the at least one feature represents a possible gunshot event in the environment. 
   
     
     
         11 . The method of  claim 1 , wherein:
 determining whether the at least one feature represents a possible gunshot event in the environment comprises:
 providing the at least one feature to the at least one classifier; 
 receiving from at least one of the classifiers a detection score that indicates a likelihood of a possible gunshot event in the environment; and 
 determining whether at least one detection score from the detection scores received from at least one of the at least one classifier indicates a detection of a likely false positive; and 
   performing the action comprises, in response to determining that at least one detection score from the detection scores received from at least one of the at least one classifier indicates a detection of a likely false positive, determining to not provide the detection of the possible gunshot event in the environment to at least one solver.   
     
     
         12 . The method of  claim 11 , wherein:
 the at least one solver comprises a muzzle solver or a flash solver, and   determining the detection of the possible gunshot event in the environment uses one or more of the muzzle solver and at least one acoustic feature from the acoustic data, or the flash solver and at least one infrared feature from the infrared data.   
     
     
         13 . The method of  claim 1 , wherein the determining whether the at least one feature represents a possible gunshot event in the environment comprises:
 detecting, in at least some of the acoustic data, a first possible gunshot event and a second possible gunshot event,   determining, using data for the first possible gunshot event and the second possible gunshot event, whether the second possible gunshot event satisfies an echo event threshold for the first possible gunshot event, and   determining, in response to determining that the second possible gunshot event satisfies the echo event threshold for the first possible gunshot event, to not provide data for the second possible gunshot event to another processing engine.   
     
     
         14 . A system comprising:
 an infrared sensor;   a plurality of classifiers;   one or more processors; and   one or more storage devices on which are stored instructions that are operable, when executed by the one or more processors, to cause the one or more processors to perform operations comprising:
 receive, from the infrared sensor, infrared data; 
 determine whether an infrared feature from the infrared data satisfies a saturation threshold; 
 select, from the plurality of classifiers, at least one classifier using a result of the determination whether the infrared feature from the infrared data satisfies the saturation threshold; 
 access at least one feature from one or more of the infrared data or acoustic data captured by an acoustic sensor; 
 determine, using the at least one classifier and the at least one feature, whether the at least one feature represents a possible gunshot event in an environment that includes the infrared sensor; and 
 perform an action using a result of the determination whether the at least one feature represents a possible gunshot event in the environment. 
   
     
     
         15 . The system of  claim 14 , comprising the acoustic sensor, and the operations comprising receiving, from the acoustic sensor, the acoustic data. 
     
     
         16 . One or more computer storage media encoded with instructions that, when executed by one or more computers, cause the one or more computers to perform operations comprising:
 determining, by a device that includes an infrared sensor and an acoustic sensor, whether infrared data captured by the infrared sensor satisfies a saturation threshold;   selecting, from a plurality of classifiers, at least one classifier using a result of the determination whether the infrared data satisfies the saturation threshold;   accessing at least one feature from one or more of the infrared data captured by the infrared sensor or acoustic data captured by the acoustic sensor;   determining, using the at least one classifier and the at least one feature, whether the at least one feature represents a possible gunshot event in an environment that includes the device; and   performing an action using a result of the determination whether the at least one feature represents a possible gunshot event in the environment.