Methods and systems of spatiotemporal pattern recognition for video content development

Abstract

Providing enhanced video content includes processing at least one video feed through at least one spatiotemporal pattern recognition algorithm that uses machine learning to develop an understanding of a plurality of events and to determine at least one event type for each of the plurality of events. The event type includes an entry in a relationship library detailing a relationship between two visible features. Extracting and indexing a plurality of video cuts from the video feed is performed based on the at least one event type determined by the understanding that corresponds to an event in the plurality of events detectable in the video cuts. Lastly, automatically and under computer control, an enhanced video content data structure is generated using the extracted plurality of video cuts based on the indexing of the extracted plurality of video cuts.

What is claimed is: 
     
         1 . A method for providing enhanced video content, comprising:
 processing at least one video feed through at least one spatiotemporal pattern recognition algorithm that uses a machine learning system to associate spatiotemporal patterns with event types and to determine at least one event type for each of a plurality of events in the at least one video feed based on at least one spatiotemporal pattern recognized in each of the plurality of events;   extracting a plurality of video cuts from the at least one video feed;   indexing the extracted plurality of video cuts based on occurrences of events represented in the plurality of video cuts that correlate to the at least one event type determined by the machine learning system;   determining at least one pattern relating to the extracted plurality of video cuts; and   indexing at least a portion of the extracted plurality of video cuts with an indicator of the pattern.   
     
     
         2 . The method of  claim 1 , wherein the at least one pattern is determined by applying machine learning. 
     
     
         3 . The method of  claim 2 , wherein using the machine learning system includes identifying at least one player involved in an event, and wherein indexing of the extracted plurality of video cuts includes identifying at least one player represented in at least one of the video cuts from the plurality of the video cuts. 
     
     
         4 . The method of  claim 2 , wherein the at least one pattern relates to a series of same event types involving a same player over time. 
     
     
         5 . The method of  claim 1 , wherein the plurality of extracted video cuts includes a player during multiple, identical event types over time. 
     
     
         6 . The method of  claim 1 , further comprising providing an enhanced video feed that shows a player during the plurality of events over time, wherein the enhanced video feed is at least one of a simultaneous, superimposed video of the player involved in multiple, identical event types and a sequential video of the player involved in a single event type. 
     
     
         7 . The method of  claim 1 , wherein determining at least one pattern includes identifying sequences of events that predict a given action that is likely to follow. 
     
     
         8 . The method of  claim 1 , wherein determining the at least one pattern includes identifying similar sequences of events across the plurality of extracted video cuts. 
     
     
         9 . The method of  claim 1 , further comprising providing a user interface that enables a user to at least one of view and interact with the at least one pattern. 
     
     
         10 . The method of  claim 9 , wherein the at least one pattern and options for the user interacting with the at least one pattern are personalized based on at least one of a user preference and a user profile. 
     
     
         11 . The method of  claim 1 , wherein the at least one pattern relates to an anticipated outcome of at least one of a game and an event within a game. 
     
     
         12 . The method of  claim 11 , further comprising providing a user with at least one of a statistic, trend information and a prediction based on the at least one pattern. 
     
     
         13 . The method of  claim 12 , wherein the at least one of the statistic, the trend information and the prediction is based on at least one of a user preference and a user profile. 
     
     
         14 . The method of  claim 1 , wherein the at least one pattern relates to play of an athlete. 
     
     
         15 . The method of  claim 14 , further comprising providing a comparison of the play of the athlete with another athlete based on a similarity of at least one of the extracted plurality of video cuts for each of the athlete and the other athlete and the at least one pattern associated with each of the athlete and the other athlete. 
     
     
         16 . The method of  claim 15 , wherein the comparison is between a professional athlete and a non-professional user. 
     
     
         17 . The method of  claim 15 , wherein the comparison is based on a similarity of a playing style of a professional athlete, as determined by a result of using the machine learning system and the at least one pattern, with at least one feature of the playing style of a non-professional user. 
     
     
         18 . The method of  claim 1 , wherein using the machine learning system further comprises using the plurality of events in position tracking data over time obtained from at least one of the at least one video feed and a chip-based player tracking system, and wherein inputs to the machine learning system include at least two of spatial configuration, relative motion, and projected motion of at least one of a player and an item used in a game. 
     
     
         19 . The method of  claim 1 , wherein using the machine learning system further comprises aligning multiple unsynchronized input feeds related to an event of the plurality of events using at least one of a hierarchy of algorithms and a hierarchy of human operators, wherein the unsynchronized input feeds are selected from the group consisting of one or more broadcast video feeds of the event, one or more feeds of tracking video for the event, and one or more play-by-play data feeds of the event. 
     
     
         20 . The method of  claim 19 , wherein the multiple unsynchronized input feeds include at least three feeds selected from at least two types related to the event. 
     
     
         21 . The method of  claim 19 , further comprising at least one of validating and modifying the alignment of the unsynchronized input feeds using a hierarchy involving at least two of one or more algorithms, one or more human operators, and one or more input feeds. 
     
     
         22 . The method of  claim 1 , further comprising at least one of validating and modifying a result of using the machine learning system by using a hierarchy involving at least two of one or more algorithms, one or more human operators, and one or more input feeds. 
     
     
         23 . The method of  claim 1 , wherein extracting the plurality of video cuts from the at least one video feed includes automatically extracting a cut from the at least one video feed using the machine learning system as applied to the plurality of events and using the machine learning system with another input feed selected from the group consisting of a broadcast video feed, an audio feed, and a closed caption feed. 
     
     
         24 . The method of  claim 23 , wherein using the machine learning system with another input feed comprising machine learning of at least one of a portion of content of a broadcast commentary and a change in camera view in the input feed. 
     
     
         25 . A system for providing enhanced video content, comprising:
 at least one video feed comprising a plurality of spatiotemporal pattern-characterized events;   a machine learning system that uses a spatiotemporal pattern recognition algorithm to associate spatiotemporal patterns with event types of spatiotemporal pattern-characterized events, the machine learning system further determining at least one event type for each of the plurality of events;   a video cut extraction circuit that receives the at least one video feed and produces a plurality of video cuts from the received at least one video feed;   a video cut indexing circuit that indexes the extracted plurality of video cuts based on occurrences of events represented in the plurality of video cuts that correlate to the at least one event type determined by the machine learning system; and   an extracted plurality of video cuts pattern detection and indexing circuit that receives a portion of the plurality of extracted video cuts, determines at least one pattern therein, and indexes at least a portion of the plurality of extracted video cuts with an indicator of the at least one pattern.