Object detection and analysis via unmanned aerial vehicle

Abstract

An unmanned aerial vehicle (UAV) can include one or more cameras for capturing image data within a field of view that depends in part upon the location and orientation of the UAV. At least a portion of the image data can be processed on the UAV to locate objects of interest, such as people or cars, and use that information to determine where to fly the drone in order to capture higher quality image data of those or other such objects. Once identified, the objects of interest can be counted, and the density, movement, location, and behavior of those objects identified. This can help to determine occurrences such as traffic congestion or unusual patterns of pedestrian movement, as well as to locate persons, fires, or other such objects. The data can also be analyzed by a remote system or service that has additional resources to provide more accurate results.

What is claimed is: 
     
         1 . A computer-implemented method for monitoring motion of objects of interest, comprising:
 capturing image data over an area of interest within a field of view of at least one camera, the image data captured over a period of time;   processing the image data to determine moving instances of the objects of interest represented in the image data and corresponding to the area of interest;   classifying the moving instances using one or more classifiers, each classifier representing a type of object and corresponding to a recorded behavior of the type of object;   determining, based at least in part on the type of object, the corresponding recorded behavior and a respective confidence level, a predicted path of motion for each of the moving instances over a subsequent period of time;   transmitting information about the predicted path of motion for the moving instances within the area of interest over the subsequent period of time.   
     
     
         2 . The computer-implemented method of  claim 1 , further comprising:
 determining a number of the moving instances of one or more types of objects represented in the image data, each of the number of moving instances having an associated confidence level based at least in part on the one or more classifiers;   updating, based at least in part on the number of moving instances and the one or more classifiers, the predicted path of motion for each of the moving instances over a subsequent period of time; and   transmitting information about the number of moving instances of the one or more types of objects of interest and the updated predicted path of motion for the area of interest over the period of time.   
     
     
         3 . The computer-implemented method of  claim 1 , further comprising:
 determining a sub-region within the area of interest where at least one of the moving instances has an associated confidence level that falls below a minimum confidence threshold; and   modifying the predicted path to cause the camera to capture more detailed image data for the at least one object of interest.   
     
     
         4 . The computer-implemented method of  claim 2 , further comprising:
 transmitting the image data to a data analysis system for subsequent analysis;   receiving, from the data analysis system, a second determination of the number of moving instances of at least one type of object of interest, the data analysis system using a deep neural network-based analysis process to generate the second determination of the number of instances; and   receiving updated training data based at least in part from the second determination.   
     
     
         5 . The computer-implemented method of  claim 1 , further comprising:
 detecting signal information of at least a subset of the moving instances of at least one type of object of interest; and   determining at least one of the number of the moving instances or the predicted path of motion based at least in part upon the signal information.   
     
     
         6 . The computer-implemented method of  claim 1 , further comprising:
 determining, based at least in part upon location data and the predicted path of motion for the moving instances of the objects of interest, an irregular object behavior;   sending a notification regarding the irregular object behavior; and   attempting to capture additional image data regarding the irregular object behavior.   
     
     
         7 . The computer-implemented method of  claim 6 , further comprising:
 sending the notification to at least one unmanned aerial vehicle (UAV); and   initiating at least one UAV to move toward a location of the irregular object behavior to capture additional image data corresponding to the irregular object behavior.   
     
     
         8 . An monitoring device, comprising:
 one or more sensors;   a processor;   a non-transitory computer readable medium storing code executable by the processor to:   capture, using the one or more sensors, sensor data for an area of interest;   recognize, by the processor, instances of objects of interest represented in the sensor data based at least in part on analyzing the sensor data, the instances recognized by a plurality of instance types each identified by a respective type classifier;   determine, based at least in part on the instances and the respective type classifier, a predicted path of motion for each of at least a subset of the instances; and   generate summary data including each instance type based on the respective type classifier and the predicted path of motions.   
     
     
         9 . The monitoring device of  claim 8 , the non-transitory computer readable medium further storing code executable by the processor to:
 capture the sensor data over a period of time;   determine the predicted path of motion for each of at least the subset of the instances over the period of time; and   generate information about the predicted path of motion over the period of time to be included in the summary data.   
     
     
         10 . The monitoring device of  claim 8 , further comprising a flight mechanism, the non-transitory computer readable medium further storing code executable by the processor to:
 detect, based on the sensor data, an event;   cause, using the flight mechanism, the monitoring device to fly along a flight path to a location of the event; and   capture, by the one or more sensors, additional sensor data along the flight path and corresponding to the location to the event.   
     
     
         11 . The monitoring device of  claim 10 , the non-transitory computer readable medium further storing code executable by the processor to:
 communicate with a second monitoring device; and   send flight instruction data to the second monitoring device, the flight instruction data causing the second monitoring device to fly to the location of the event or capture a second set of sensor data with respect to the area of interest.   
     
     
         12 . The monitoring device of  claim 10 , the non-transitory computer readable medium further storing code executable by the processor to:
 capture a second set of sensor data using at least a second sensor of the one or more sensors of the monitoring device, the second set of sensor data including at least one of heat signature data, radio frequency data, audio data, orientation data, or motion data;   determine, based at least in part on the second set of sensor data, a number of the instances or a respective location of each of the instances of the objects within the area of interest; and   modify the predicted path of motion of the instances or the flight path of the monitoring device.   
     
     
         13 . The monitoring device of  claim 10 , the non-transitory computer readable medium further storing code executable by the processor to:
 modify the flight path of the monitoring device based at least in part upon at least one of a density, a movement, a number, an event, or an unexpected action of at least a subset of the objects of interest.   
     
     
         14 . The monitoring device of  claim 8 , wherein the sensor data includes at least one of heat signature data, radio frequency data, audio data, motion data, ambient light image data, infrared image data, or stereoscopic image data. 
     
     
         15 . The monitoring device of  claim 8 , wherein the generated summary data is transmitted, stored, downloaded, or uploaded to another monitoring device, a computing device, or a server 
     
     
         16 . A method of monitoring objects of interest in a predetermined area, comprising:
 determining a first flight path of a first monitoring device having a first camera to capture first image data of the predetermined area over a period of time for the first monitoring device to traverse the first flight path;   analyzing the first image data to detect objects of interest located within the predetermined area;   classifying the objects of interest into a plurality of types, each type identified by a respective type classifier;   determining, based at least in part on a number of the objects of interest and the respective type classifier, a predicted path of motion for the objects of interest;   modifying , based at least in part upon the respective type classifier, a location, or the predicted path of motion of the objects of interest, the flight path the first monitoring device to capture additional image data of the predetermined area; and   generating report data including the number of the objects of interest, the respective type classifier, one or more locations or positions of the objects of interest, a density, or the predicted path of motion of the objects of interest within the predetermined area over the period of time.   
     
     
         17 . The method of  claim 16 , further comprising:
 transmitting the image data to a processing system comprising a database of image data, the processing system using object detection algorithm to detect and classify the objects of interest represented in the image data.   
     
     
         18 . The method of  claim 16 , wherein the report data includes at least one of types of traffic, congestion points, patterns of motion, changes in traffic patterns over time, different paths of motion for different types of objects, unusual movements, and unexpected occurrences. 
     
     
         19 . The method of  claim 16 , wherein the report data is transmitted, downloaded, stored, or uploaded to another monitoring device, a computing device, or a server 
     
     
         20 . The method of  claim 16 , further comprising:
 determining a second flight path of a second monitoring device having a second camera to capture second image data of the predetermined area over the period of time for the second monitoring device to traverse the second flight path;   aggregating the second image data with the first image data;   analyzing the aggregated second image data and the first image data to detect the objects of interest located within the predetermined area;   classifying, based on analyzing the aggregated second image data and the first image data, the objects of interest into the plurality of types, each type identified by the respective type classifier; and   generating a multi-dimensional representation of at least a portion of the predetermined area and the objects of interest within the predetermined area.