Video summary description scheme and method and system of video summary description data generation for efficient overview and browsing

Abstract

A video summary description scheme for describing video summary intervals by meta data that provides overview functionality, and which makes it feasible to understand overall contents of the original video within a short time, and navigation and browsing functionalities, which make it feasible to search the desired video contents efficiently. A Video Summary DS includes at least one HighlightSegment DS that describes information on a highlight segment corresponding to one of the video summary intervals, wherein the HighlightSegment DS comprises a VideoSegmentLocator DS describing the highlight segment and an ImageLocator DS describing a representative frame of the highlight segment.

1 . A computer-readable recording medium having a Video Summary Description Scheme (DS) for describing a video summary interval stored therein, the Video Summary DS comprising: at least one HighlightSegment DS for describing information about a highlight segment corresponding to the video summary interval, wherein the HighlightSegment DS comprises a VideoSegmentLocator DS for describing the highlight segment and an ImageLocator DS for describing a representative frame of the highlight segment.  
   
   
       2 . The computer-readable recording medium of  claim 1  wherein the VideoSegmentLocator DS comprises one of time information and video itself of the highlight segment.  
   
   
       3 . A method for generating video summary description data corresponding to original video according to a video summary description scheme, comprising the steps of: 
 (a) analyzing the original video and producing a video analysis result;    (b) defining a summary rule for selecting a video summary interval;    (c) selecting the video summary interval capable of summarizing video contents from the original video based on the original video analysis result and the summary rule, which constitute video summary interval information;    (d) extracting a representative frame based on the video summary interval information; and    (e) generating video summary description data according to a Video Summary Description Scheme (DS) for enabling execution of browsing based on the video summary interval information and the representative frame,    wherein the Video Summary DS comprises at least one HighlightSegment DS for describing information on a highlight segment corresponding to the video summary interval, wherein the HighlightSegment DS comprises a VideoSegmentLocator DS for describing the highlight segment and an ImageLocator DS for describing a representative frame of the highlight segment.    
   
   
       4 . The method of  claim 3  wherein the VideoSegmentLocator DS comprises one of time information and video itself of the highlight segment.  
   
   
       5 . The method of  claim 4  wherein step (a) comprises the steps of: 
 extracting features from the original video and outputting the types of features and video time interval at which those features are detected;    detecting key events included in the original video based on the types of features and video time interval at which those features are detected; and    detecting an episode by dividing the original video into a story flow base on the basis of the detected key events.    
   
   
       6 . The method of  claim 4  wherein step (d) comprises the step of extracting a representative sound from the video summary interval information.  
   
   
       7 . The method of  claim 4  wherein the HighlightSegment DS further comprises a SoundLocator DS describing representative sound information of the highlight segment.  
   
   
       8 . The method of  claim 4  wherein the HighlightSegment DS further comprises an AudioSegmentLocator DS describing audio segment information constituting an audio summary of the highlight segment.  
   
   
       9 . A system for generating video summary description data of original video according to a video summary description scheme, comprising: 
 video analyzing means for analyzing the original video and producing a video analysis result;    summary rule defining means for defining a summary rule for selecting a video summary interval;    video summary interval selecting means for selecting a video interval capable of summarizing the video contents of the original video and outputting video summary interval information based on the video analysis result from the video analyzing means and the summary rule from the summary rule defining means;    representative frame extracting means for outputting a representative frame representing the video summary interval based on the video summary interval information from the video summary interval selecting means; and    video summary describing means for generating video summary description data with a Video Summary Description Scheme (DS) by inputting the video summary interval information from the video summary interval selecting means and the representative frame information from the representative frame extracting means,    wherein the Video Summary DS comprises at least one HighlightSegment DS for describing information on a highlight segment corresponding to the video summary interval, wherein the HighlightSegment DS comprises a VideoSegmentLocator DS for describing the highlight segment and an ImageLocator DS for describing a representative frame of the highlight segment.    
   
   
       10 . The system of  claim 9  wherein the VideoSegmentLocator DS comprises one of time information and video itself of the highlight segment.  
   
   
       11 . The system of  claim 10  wherein the video analyzing means comprises: 
 feature extracting means for extracting features from the original video and producing types of features and a video time interval at which the types of features are detected;    event detecting means for detecting key events included in the original video by inputting the types of features and the video time interval at which the types of features are detected; and    episode detecting means for detecting an episode by dividing the original video into a story flow base on the basis of the detected event.    
   
   
       12 . The system of  claim 10 , the system further comprises representative sound extracting means for extracting a representative sound by inputting the video summary interval information and providing the extracted representative sound to a video summary describing means.  
   
   
       13 . The system of  claim 10  wherein the HighlightSegment DS further comprises a SoundLocator DS for describing a representative sound information of the highlight segment.  
   
   
       14 . The system of  claim 10  wherein the HighlightSegment DS further comprises an AudioSegmentLocator DS for describing audio segment information constituting an audio summary of the highlight segment.  
   
   
       15 . An apparatus for browsing video summary description data, the video summary description data having a Video Summary Description Scheme (DS) for describing video summary intervals, the Video Summary DS comprising at least one HighlightSegment DS for describing information on a highlight segment corresponding to one of the video summary intervals, the HighlightSegment DS comprising a VideoSegmentLocator DS for describing the highlight segment and an ImageLocator DS for describing a representative frame of the highlight segment.  
   
   
       16 . The apparatus of  claim 15  wherein the apparatus is arranged to display a representative frame of the highlight segment on a display device and to play the highlight segment.  
   
   
       17 . The apparatus of  claim 16  wherein the VideoSegmentLocator DS describes one of time information and video itself of the highlight segment.  
   
   
       18 . The apparatus of  claim 17  wherein the HighlightSegment DS further comprises: 
 a SoundLocator DS for describing representative sound information of the highlight segment; and    an AudioSegmentLocator DS for describing audio segment information constituting an audio summary of the highlight segment.    
   
   
       19 . An apparatus for browsing video summary description data corresponding to an original video, the video summary description data having a HierarchicalSummary Description Scheme (DS) for describing a video summary, the apparatus comprising: 
 a video player for playing an original video or the video summary;    an original video representative frame player for playing a representative frame of the original video; and    a video summary representative frame player for playing a summary level of video interval.    
   
   
       20 . The apparatus of  claim 19  wherein the HierarchicalSummary DS comprises a HighlightLevel DS that comprises at least one HighlightSegment DS describing information on a highlight segment corresponding to the video summary interval, 
 the HighlightSegment DS comprising a VideoSegmentLocator DS for describing the highlight segment, and an ImageLocator DS for describing a representative frame of the highlight segment.    
   
   
       21 . The apparatus of  claim 20  wherein the VideoSegmentLocator DS describes one of time information and video itself of the highlight segment.  
   
   
       22 . The apparatus of  claim 21  wherein the HighlightSegment DS further comprises: 
 a SoundLocator DS for describing representative sound information of the highlight segment; and    an AudioSegmentLocator DS for describing an audio segment information constituting an audio summary of the highlight segment.    
   
   
       23 . A Video Summary Description Scheme (DS) for describing video summary intervals of an original video, wherein the Video Summary DS comprises at least one HighlightSegment DS for describing information on a highlight segment corresponding to one of video summary intervals, wherein the HighlightSegment DS comprises a VideoSegmentLocator DS for describing the highlight segment and an ImageLocator DS for describing a representative frame of the highlight segment.