Method and system for classifying scene for each person in video

Abstract

Described is a method of classifying a scene for each person in a video, the method including: detecting a face within input video frames; detecting a shot change of the input video frames; extracting a person representation frame in the shot; performing a person clustering in the extracted person representation frame based on time information; detecting a scene change by separating a person portion from a background based on face extraction information, and comparing the person portion and the background; and merging similar clusters from the extracted person representation frame and performing a scene clustering for each person.

What is claimed is: 
     
         1 . A method of classifying a scene for each person in a video, the method comprising:
 extracting a person representation frame in a shot;   comparing a first person representation frame and a second person representation frame;   performing a person clustering by extending a time window when the first person representation frame is similar to the second person representation frame;   merging similar clusters using a person cluster extracted from a representation frame and performing a scene clustering for each person based on a scene change,   wherein the scene change is determined using a person portion and a background portion.   
     
     
         2 . The method of  claim 1 , wherein the performing of the person clustering further comprises:
 receiving cluster information, the first person representation frame, and the second person representation frame to be compared;   including the second person representation frame which has been currently compared in the current cluster information when the first person representation frame is similar to the second person representation frame; and   setting a subsequent person representation frame as third person representation frame to be compared on the time window.   
     
     
         3 . The method of  claim 2 , further comprising:
 moving to a subsequent cluster when the first person representation frame and the second person representation frame are at the end of the time window; or   setting the subsequent person representation frame in the time window as the other person representation frame to be compared on the time window, when the first person representation frame and the second person representation frame to be compared are not at the end of the time window.   
     
     
         4 . The method of  claim 1 , wherein the performing of the scene clustering comprises:
 receiving time information-based clusters;   selecting two clusters having a minimum difference value;   comparing the minimum difference value and a threshold value; and   merging the two clusters when the minimum difference value is less than the threshold value.   
     
     
         5 . A non-transitory computer-readable recording medium storing a program for implementing a method of classifying a scene for each person in a video, the method comprising:
 extracting a person representation frame in a shot;   comparing a first person representation frame and a second person representation frame;   performing a person clustering by extending a time window when the first person representation frame is similar to the second person representation frame;   merging similar clusters using a person cluster extracted from a representation frame and performing a scene clustering for each person based on a scene change,   wherein the scene change is determined using a person portion and a background portion.   
     
     
         6 . A system for classifying a scene for each person in a video, the system comprising:
 a person representation frame extracting unit to extract a person representation frame in a shot;   a person clustering unit to compare a first person representation frame and a second person representation frame and to perform a person clustering by extending a time window when the first person representation frame is similar to the second person representation frame;   a scene clustering unit to merge similar clusters using a person cluster extracted from a representation frame and to perform a scene clustering for each person based on a scene change,   wherein the scene change is determined using a person portion and a background portion.