Optimizations for live event, real-time, 3d object tracking

Abstract

A system for automatically providing event non-video information usable for indexing event video information, where the event takes place at an event location for a duration of event time. The system includes a non-video information system for receiving or determining event non-video information and storing the non-video information in a non-video dataset, a video information system for receiving event video information captured by one or more filming cameras and storing the video information in a video dataset, and a video information retrieval system for randomly accessing the video dataset, where the random access includes: (a) using event non-video information to first determine an event time or duration of event time, and (b) using the determined event time or duration of event time to selectively access, retrieve, or provide event video information including one or more video images of the event.

What is claimed: 
     
         1 . A system for automatically providing event non-video information usable for indexing event video information, where the event takes place at an event location for a duration of event time, comprising:
 a non-video information system for receiving or determining event non-video information, where the event non-video information relates to event activities taking place at the event location during the event time, where the event activities comprise the movements of one or more event persons or event objects, and where the event non-video information is relatable to the event time and is stored in a non-video dataset;   a video information system for receiving event video information captured by one or more filming cameras, where the event video information records at least some of the event activities taking place at the event location during the event time, and where the event video information is relatable to event time and is stored in a video dataset; and   a video information retrieval system for randomly accessing the video dataset, where the video information retrieval system has access to both the non-video dataset and the video dataset, where the random access includes: (a) using event non-video information to first determine an event time or duration of event time, and (b) using the determined event time or duration of event time to selectively access, retrieve, or provide event video information including one or more video images of the event.   
     
     
         2 . The system of  claim 1  where the event non-video information includes any one of, or any combination of event quantifications relating to the event or one or more event persons or event objects, where event quantifications include any one of, or any combination of object tracking information or event statistics, where the object tracking information includes any one of, or any combination of the on-going location, orientation, trajectory, acceleration or velocity of one or more event persons or event objects, and where if the event is a sporting event, event persons are athletes or game officials, event objects are game objects such as a puck in the sport of ice hockey, and event statistics at least include any one of, or any combination of: playing time, playing time per event sub-locations, average speed, time in control of the game object, number of scoring attempts, turnovers, steals or passes. 
     
     
         3 . The system of  claim 2  where one or more identifying items are affixed to one or more event persons or event objects prior to the event time, where the identifying items are usable for determining identifying item information that is non-video information, and where the event quantifications are determined in part from the identifying item information. 
     
     
         4 . The system of  claim 3  where the one or more identifying items are marks detectable as video data, where the identifying item marks are represented within tracking video information captured by one or more tracking cameras, where the tracking video information is distinct from the event video information, where the tracking video information is usable for determining the identifying item information by performing image analysis on the tracking video information in order to locate and quantify the identifying item information. 
     
     
         5 . The system of  claim 4  where the one or more identifying marks are any one of, or any combination of non-visible or encoded, where encoded marks include bar codes or similar formats. 
     
     
         6 . The system of  claim 3  where the object tracking information determined from the identifying item information includes three dimensional representations of one or more event persons or event objects, where the three dimensional representations are used to generate event animation information, and where the event video information stored in the video dataset is any combination of the event video information captured from cameras and the event animation information generated from the object tracking information. 
     
     
         7 . The system of  claim 6  where the video information retrieval system is further enabled to retrieve any combination of the event video information, event animation information or event non-video information either local to the event or remote to the event, where the time of access is either during the event time or after the event time, and where the time of access during the event time is either real-time or post real-time. 
     
     
         8 . The system of  claim 2  where the event non-video information comprises in part object tracking information, where the object tracking information is determined in part by performing image analysis on tracking video information captured from one or more tracking cameras, where the tracking video information is distinct from the event video information and includes event activities being performed during event time. 
     
     
         9 . The system of  claim 8  where the tracking video information is usable as event video information. 
     
     
         10 . The system of  claim 8  where one or more identifying marks are affixed to one or more event persons or event objects prior to the event time, where the identifying marks are usable for determining identifying mark information that is non-video information, where the identifying marks are detectable as video data represented within the tracking video information, where the tracking video information is usable for determining the identifying mark information by performing image analysis on the tracking video information in order to locate and quantify the identifying mark information, and where the event quantifications are determined in part from the identifying mark information. 
     
     
         11 . The system of  claim 1  where the event non-video information includes pre-event non-video information determined prior to the duration of event time and live-event non-video information determined during the duration of event time, where the pre-event non-video information includes event person group information or event location information, where the event location information includes layout information describing the event area where the event activities take place, where the event is a sporting event and the person group information includes a list of players and team associations if the sporting event is a team sport, and where the live-event non-video information includes any one of, or any combination of the event quantifications. 
     
     
         12 . The system of  claim 1  where the non-video information system, the video information system and the video information retrieval system operate in real-time such that the random access of the event video information occurs either in real-time or post real-time. 
     
     
         13 . The system of  claim 1  where individuals access and retrieve any one of, or any combination of the event video information and the event non-video information either local to the event or remote to the event, where the time of access is either during the event time or after the event time, and where the time of access during the event time is either real-time or post real-time. 
     
     
         14 . The system of  claim 1  where any one of, or any combination of: 1) the non-video information system, 2) the video information system, or 3) the video information retrieval system, comprise either different systems or the same system. 
     
     
         15 . The system of  claim 2  where at least one of the one or more filming cameras for capturing event video information are adjustable filming cameras, where the adjustments include any one of, or any combination of changes in the pan, tilt or zoom setting of the adjustable filming camera, where each adjustable filming camera is further adapted to receive electronic adjustment signals and to make corresponding adjustments, and where the electronic adjustment signals are generated in response to the event quantifications. 
     
     
         16 . The system of  claim 15  where event quantifications including object tracking data are used to assign or re-assign any adjustable filming camera to film any event persons. 
     
     
         17 . The system of  claim 1  where the event video information system has access to event location background image information corresponding to the captured event video information, where the event video information system compares the captured event video information to the corresponding background image information in order to create a minimal set of foreground pixels, and where the event video information stored in the video dataset is limited based at least in part upon the determined minimal set of foreground pixels. 
     
     
         18 . The system of  claim 17  where the video information retrieval system has access to the background image information corresponding to the video dataset of foreground limited event video information, where the video information retrieval system recreates non-limited event video information comprising the minimal set of foreground pixels combined with the corresponding background image information. 
     
     
         19 . The system of  claim 17  where the event video information system analyzes the minimal set of foreground pixels to create a stylized graphical representation of the event foreground, and where the event video information stored in the video dataset includes the stylized graphical representation. 
     
     
         20 . The system of  claim 17  where the video information retrieval system presents a graphical representation of the randomly accessed event video information stored in the video dataset using at least in part the stylized graphical representations of the event foreground contained within the video dataset. 
     
     
         21 . A system for allowing a user to interactively retrieve event video information by first randomly accessing event non-video information, comprising:
 a data synchronizer operative on a computer for receiving or accessing event non-video information and event video information, where the data synchronizer cross-links the event non-video dataset with the event video dataset using event time such that the event non-video information is relatable to the event video information by event time, and   a video information retrieval system operative on a computer for randomly accessing the event video information using the synchronized non-video information, where the random access includes: (a) using event non-video information to first determine an event time or duration of event time, and (b) using the determined event time or duration of event time to selectively access, retrieve, or provide event video information including one or more video images of the event.   
     
     
         22 . The system of  claim 21  where the event non-video information includes any one of, or any combination of event quantifications relating to the event or one or more event persons or event objects, where event quantifications include any one of, or any combination of object tracking information or event statistics, where the object tracking information includes any one of, or any combination of the on-going location, orientation, trajectory, acceleration or velocity of one or more event persons or event objects, and where if the event is a sporting event, event persons are athletes or game officials, event objects are game objects such as a puck in the sport of ice hockey, and event statistics at least include any one of, or any combination of: playing time, playing time per event sub-locations, average speed, time in control of the game object, number of scoring attempts, turnovers, steals or passes. 
     
     
         23 . The system of  claim 22  where one or more identifying items are affixed to one or more event persons or event objects prior to the event time, where the identifying items are usable for determining identifying item information that is non-video information, where the event quantifications are determined in part from the identifying item information, and where the data synchronizer additionally receives or accesses the identifying item information as additional non-video information. 
     
     
         24 . The system of  claim 22  where the event non-video information includes pre-event non-video information determined prior to the duration of event time and live-event non-video information determined during the duration of event time, where the pre-event non-video information includes event person group information or event location information, where the event location information includes layout information describing the event area where the event activities take place, where the event is a sporting event and the person group information includes a list of players and team associations if the sporting event is a team sport, and where the live-event non-video information includes any one of, or any combination of the event quantifications. 
     
     
         25 . The system of  claim 21  where the data synchronizer and the video information retrieval system operate in real-time such that the random access of the event video information occurs either in real-time or post real-time. 
     
     
         26 . A system for creating a non-video dataset in accordance with an event including event non-video information relatable to event time for use in providing random access to a video dataset captured in accordance with the event including event video information relatable to event time, comprising:
 a non-video information system for receiving or determining event non-video information, where the event non-video information relates to event activities taking place at an event location during event time, where the event activities comprise the movements of one or more event persons or event objects, and   a non-video dataset, where the non-video dataset includes the event non-video information stored in relation to event time, where the non-video dataset is relatable to a video dataset stored in relation to event time, where the video dataset includes event video information, where the event video information is captured by one or more filming cameras, where the event video information records at least some of the event activities taking place at the event location during the event time and is relatable to event time, where the non-video data is adapted to be used by a video information retrieval system in order to randomly access the video dataset, where the video information retrieval system has access to both the non-video dataset and the video dataset, and where the random access includes: (a) using event non-video information to first determine an event time or duration of event time, and (b) using the determined event time or duration of event time to selectively access, retrieve, or provide event video information including one or more video images of the event.   
     
     
         27 . The system of  claim 26  where the event non-video information includes any one of, or any combination of event quantifications relating to the event or one or more event persons or event objects, where event quantifications include any one of, or any combination of object tracking information or event statistics, where the object tracking information includes any one of or any combination of the on-going location, orientation, trajectory, acceleration or velocity of one or more event persons or event objects, and where if the event is a sporting event, event persons are athletes or game officials, event objects are game objects such as a puck in the sport of ice hockey, and event statistics at least include any one of, or any combination of: playing time, playing time per event sub-locations, average speed, time in control of the game object, number of scoring attempts, turnovers, steals or passes. 
     
     
         28 . The system of  claim 27  where one or more identifying items are affixed to one or more event persons or event objects prior to the event time, where the identifying items are usable for determining identifying item information that is non-video information, where the event quantifications are determined in part from the identifying item information, and where the video information retrieval system additionally receives or accesses the identifying item information as additional non-video information. 
     
     
         29 . The system of  claim 26  where the event non-video information includes pre-event non-video information determined prior to the duration of event time and live-event non-video information determined during the duration of event time, where the pre-event non-video information includes event person group information or event location information, where the event location information includes layout information describing the event area where the event activities take place, where the event is a sporting event and the person group information includes a list of players and team associations if the sporting event is a team sport, and where the live-event non-video information includes any one of, or any combination of the event quantifications. 
     
     
         30 . The system of  claim 26  where the non-video information system and the video information retrieval system operate in real-time such that the random access of the event video information occurs either in real-time or post real-time.