System and method for enhanced video image recognition using motion sensors

Abstract

Disclosed are systems and methods for improving image recognition by using information from sensor data. In one embodiment, the method comprises receiving one or more sensor records, the sensor records representing timestamped sensor data collected by a sensor recording device; selecting an event based on the sensor records; identifying a time associated with the event; retrieving a plurality of timestamped video frames; synchronizing the sensor records and the video frames, wherein synchronizing the sensor records and the video frames comprises synchronizing the timestamped sensor data with individual frames of the timestamped video frames according to a common timeframe; and selecting a subset of video frames from the plurality of timestamped video frames based on the selected event.

What is claimed is: 
     
         1 . A method of calibrating an image capture system, comprising:
 receiving a plurality of video frames with metadata, wherein the plurality of video frames are associated with an event;   selecting timestamped performance data corresponding with the plurality of video frames based on the metadata;   synchronizing timestamped performance data and the plurality of timestamped video frames associated with the event by synchronizing the timestamped performance data with individual frames from the plurality of timestamped video frames;   identifying a portion of each video frame in which a target is present; and   calibrating at least one parameter of at least one camera based on the portion of each video frame in which a target is present.   
     
     
         2 . The method of  claim 1 , wherein identifying a portion of each video frame in which a target is present comprises identifying pixels in each video frame in which the target is present. 
     
     
         3 . The method of  claim 1 , wherein identifying a portion of each video frame in which a target is present comprises identifying pixels in each video frame in which the target is present via image recognition. 
     
     
         4 . The method of  claim 1 , wherein identifying a portion of each video frame in which a target is present comprises identifying pixels in each video frame in which the target is present semi-automatically by providing at least one pixel in a first frame and using image recognition for each subsequent video frame. 
     
     
         5 . The method of  claim 1 , wherein calibrating at least one parameter of at least one camera comprises calibrating a field of view of the at least one camera. 
     
     
         6 . The method of  claim 5 , wherein calibrating the field of view of the at least one camera comprises calibrating the field of view of the at least one camera via using pairs of sensor coordinates and video frame pixel locations for the event. 
     
     
         7 . The method of  claim 1 , wherein selecting timestamped performance data corresponding with the plurality of video frames comprises selecting timestamped performance data based on user identification present within the video metadata. 
     
     
         8 . The method of  claim 1 , wherein selecting timestamped performance data corresponding with the plurality of video frames comprises selecting timestamped performance data having the same time and location tag as the plurality of video frames. 
     
     
         9 . The method of  claim 1 , wherein receiving a plurality of video frames with metadata comprises receiving a plurality of video frames with metadata comprising information related to the plurality of video frames. 
     
     
         10 . The method of  claim 1 , wherein receiving a plurality of video frames with metadata comprises receiving a plurality of video frames with metadata comprising information related to at least one participant in at least one of the plurality of video frames. 
     
     
         11 . The method of  claim 1 , further comprising processing the metadata by extracting the video metadata from the video metadata. 
     
     
         12 . A system for calibrating an image capture system, comprising:
 a sensor recording device configured to capture at least one sensor record, the at least one sensor record representing timestamped sensor data collected by the sensor recording device;   at least one camera configured to record a plurality of timestamped video frames; and   an event processing system configured to perform the following instructions:
 receive a plurality of video frames with metadata, wherein the plurality of video frames are associated with an event; 
 select timestamped performance data corresponding with the plurality of video frames based on the metadata; 
 synchronize timestamped performance data and the plurality of timestamped video frames associated with the event by synchronizing the timestamped sensor data with individual frames from the plurality of timestamped video frames; 
 identify a portion of each video frame in which a target is present; and 
 calibrate at least one parameter of the at least one camera based on the portion of each video frame in which a target is present. 
   
     
     
         13 . The system of  claim 12 , wherein identify a portion of each video frame in which a target is present comprises identifying pixels in each video frame in which the target is present. 
     
     
         14 . The system of  claim 12 , wherein identify a portion of each video frame in which a target is present comprises identifying pixels in each video frame in which the target is present via image recognition. 
     
     
         15 . The system of  claim 12 , wherein identify a portion of each video frame in which a target is present comprises identifying pixels in each video frame in which the target is present semi-automatically by providing at least one pixel in a first frame and using image recognition for each subsequent video frame. 
     
     
         16 . The system of  claim 12 , wherein calibrate at least one parameter of the at least one camera comprises calibrating a field of view of the at least one camera via using pairs of sensor coordinates and video frame pixel locations for the event. 
     
     
         17 . The system of  claim 12 , wherein select timestamped performance data corresponding with the plurality of video frames comprises selecting timestamped performance data based on user identification present within the video metadata. 
     
     
         18 . A method of calibrating an image capture system, comprising:
 receiving a plurality of video frames with metadata, wherein the plurality of video frames are associated with an event;   selecting timestamped performance data corresponding with the plurality of video frames based on the metadata;   synchronizing timestamped performance data and the plurality of timestamped video frames associated with the event by synchronizing the timestamped sensor data with individual frames from the plurality of timestamped video frames;   selecting a subset of video frames from the plurality of timestamped video frames based on the event;   identifying a portion of each video frame in which a target is present; and   calibrating at least one parameter of at least one camera based on the portion of each video frame in which a target is present.   
     
     
         19 . The method of  claim 18 , wherein identifying a portion of each video frame in which a target is present comprises identifying pixels in each video frame in which the target is present via image recognition. 
     
     
         20 . The method of  claim 1 , wherein calibrating at least one parameter of the at least one camera comprises calibrating a field of view of the at least one camera via using pairs of sensor coordinates and video frame pixel locations for the event.