Multi-modal collaborative web-based video annotation system

Abstract

A video annotation interface includes a video pane configured to display a video, a video timeline bar including a video play-head indicating a current point of the video which is being played, a segment timeline bar including initial and final handles configured to define a segment of the video for playing, and a plurality of color-coded comment markers displayed in connection with the video timeline bar. Each of the comment markers is associated with a frame or segment of the video and corresponds to one or more annotations for that frame or segment made by one of a plurality of users. Each of the users can make annotations and view annotations made by other users. The annotations can include annotations corresponding to a plurality of modalities, including text, drawing, video, and audio modalities.

What is claimed is: 
     
         1 . A video annotation software product stored in a memory of an electronic device and executed by a processor causing the processor to perform the steps comprising:
 receiving data corresponding to a video, data corresponding to a plurality of point comments associated with the video, each of the plurality of point comments being associated with a single point in time of the video and with a particular user account of a plurality of user accounts, and data corresponding to a plurality of segment comments associated with the video, each of the plurality of segment comments being associated with a span of time of the video and with a particular user account of the plurality of user accounts;   displaying to a user a video annotation interface comprising a video pane configured to display the video, a first video timeline bar including a video play-head indicating a current point of the video which is being played, a second segment timeline bar including initial and final handles configured to define a segment of the video for playing, a plurality of point comment markers, each of the first plurality of comment markers corresponding to one of the plurality of point comments associated with the video, a plurality of segment comment markers, each of the second plurality of comment markers corresponding to one of the plurality of segment comments associated with the video, a comment display pane displaying text corresponding to at least some of the plurality of comments associated with the video, and a comment control configured to allow the user to add a comment to the video;   receiving, from the user, input corresponding to one or more desired annotations;   associating the input one or more annotations with a selected first segment of the video;   updating the video annotation interface so that the plurality of comment markers displayed in connection with the video timeline bar includes a new segment comment marker corresponding to the first segment; and   displaying an indication of the input one or more annotations overlaid over the video in the video pane as an annotated video.   
     
     
         2 . The software product of  claim 1 , wherein the received data corresponding to the plurality of point comments associated with the video includes data corresponding to a particular point comment which includes an identification of a point of the video associated with the comment. 
     
     
         3 . The software product of  claim 2 , wherein the identification of the point of the video comprises a timestamp. 
     
     
         4 . The software product of  claim 2 , wherein the identification of the point of the video comprises a frame identification. 
     
     
         5 . The software product of  claim 1 , wherein the received data corresponding to the plurality of segment comments associated with the video includes data corresponding to a particular segment comment, and wherein the data corresponding to the particular segment comment includes an identification of a starting point of a segment of the video associated with the particular segment comment. 
     
     
         6 . The software product of  claim 5 , wherein the data corresponding to the particular segment comment includes an identification of a length of the segment comment. 
     
     
         7 . The software product of  claim 5 , wherein the data corresponding to the particular segment comment includes an identification of an end point of the segment of the video associated with the particular segment comment. 
     
     
         8 . The software product of  claim 1 , wherein the steps further comprise receiving, from the user, input corresponding to a baseline line segment on the video pane and input representing a baseline measurement for the baseline line segment. 
     
     
         9 . The software product of  claim 1 , wherein each of the plurality of comment markers is color coded based on the particular user account associated with a comment that the comment marker is associated with. 
     
     
         10 . The software product of  claim 1 , wherein the one or more annotations include a video annotation. 
     
     
         11 . The software product of  claim 1 , wherein the one or more annotations includes an audio annotation. 
     
     
         12 . The software product of  claim 1 , wherein the one or more annotations includes a drawing annotation. 
     
     
         13 . The software product of  claim 1 , wherein the steps further comprise loading a key code library representing an association of keys with one or more respective desired annotations, and wherein receiving, from the user, the input corresponding to the one or more desired annotations comprises receiving input corresponding to a particular key which is associated with one or more respective desired annotations in the key code library. 
     
     
         14 . A video annotation method comprising:
 receiving data corresponding to a video, data corresponding to a plurality of point comments associated with the video, each of the plurality of point comments being associated with a single point in time of the video and with a particular user account of a plurality of user accounts, and data corresponding to a plurality of segment comments associated with the video, each of the plurality of segment comments being associated with a span of time of the video and with a particular user account of the plurality of user accounts;   displaying to a user a video annotation interface comprising a video pane configured to display the video, a first video timeline bar including a video play-head indicating a current point of the video which is being played, a second segment timeline bar including initial and final handles configured to define a segment of the video for playing, a plurality of point comment markers, each of the first plurality of comment markers corresponding to one of the plurality of point comments associated with the video, a plurality of segment comment markers, each of the second plurality of comment markers corresponding to one of the plurality of segment comments associated with the video, a comment display pane displaying text corresponding to at least some of the plurality of comments associated with the video, and a comment control configured to allow the user to add a comment to the video;   receiving, from the user, input corresponding to one or more desired annotations;   associating the input one or more annotations with a selected first segment of the video;   updating the video annotation interface so that the plurality of comment markers displayed in connection with the video timeline bar includes a new segment comment marker corresponding to the first segment; and   displaying an indication of the input one or more annotations overlaid over the video in the video pane as an annotated video.   
     
     
         15 . The method of  claim 14 , wherein the received data corresponding to the plurality of point comments associated with the video includes data corresponding to a particular point comment which includes an identification of a point of the video associated with the comment. 
     
     
         16 . The method of  claim 15 , wherein the identification of the point of the video comprises a timestamp. 
     
     
         17 . The method of  claim 15 , wherein the identification of the point of the video comprises a frame identification. 
     
     
         18 . The method of  claim 14 , wherein the received data corresponding to the plurality of segment comments associated with the video includes data corresponding to a particular segment comment, and wherein the data corresponding to the particular segment comment includes an identification of a starting point of a segment of the video associated with the particular segment comment. 
     
     
         19 . The method of  claim 18 , wherein the data corresponding to the particular segment comment includes an identification of a length of the segment comment. 
     
     
         20 . The method of  claim 18 , wherein the data corresponding to the particular segment comment includes an identification of an end point of the segment of the video associated with the particular segment comment. 
     
     
         21 . The method of  claim 14 , further comprising receiving, from the user, input corresponding to a baseline line segment on the video pane and input representing a baseline measurement for the baseline line segment. 
     
     
         22 . The method of  claim 14 , further comprising loading a key code library representing an association of keys with one or more respective desired annotations, and wherein receiving, from the user, the input corresponding to the one or more desired annotations comprises receiving input corresponding to a particular key which is associated with one or more respective desired annotations in the key code library.