Apparatus and method for configuring screen for video call using facial expression

Abstract

Apparatus and method for configuring a screen for a video call using a facial expression by recognizing a face from an image, calculating facial expression information for an expression of the recognized face, and determining whether there is a change in expression of the recognized face by comparing the calculated facial expression information with reference expression information preset to determine a change in expression of the face. If there is a change in expression of the recognized face, the apparatus and method selects a video image corresponding to the changed expression in the video call screen, and reconfigures the video call screen using the selected video image, making it possible for a user to conveniently select an image of the interested person without taking extensive action, and preventing a wrong image from being selected due to the unintended user facial movement.

What is claimed is: 
     
         1 . An apparatus for configuring a screen for a video call using a facial expression, comprising:
 a facial expression information calculator for recognizing a face from an image, and calculating facial expression information for an expression of the recognized face;   a facial expression determiner for determining whether there is a change in expression of the recognized face by comparing the calculated facial expression information with reference expression information preset to determine a change in expression of the face;   a screen configurer for configuring a video call screen including multiple video images received for the video call; and   an image selector for selecting a video image corresponding to the changed expression in the video call screen if there is a change in expression,   wherein the screen configurer reconfigures the video call screen using the selected video image.   
     
     
         2 . The apparatus of  claim 1 , wherein the video call screen includes a screen of a main image received from a camera and a screen of at least one sub image received outside of the video call. 
     
     
         3 . The apparatus of  claim 2 , wherein the facial expression information calculator recognizes a face in the main image, extracts facial features of the recognized face, and calculates a face angle of the recognized face based on the extracted facial features. 
     
     
         4 . The apparatus of  claim 3 , wherein the facial expression information calculator calculates a face angle from an image received before the video call, and sets the calculated face angle as a reference face angle. 
     
     
         5 . The apparatus of  claim 4 , wherein the facial expression determiner determines whether a difference between the calculated face angle and the reference face angle is greater than or equal to a preset threshold by comparing the calculated face angle with the reference face angle. 
     
     
         6 . The apparatus of  claim 5 , wherein the image selector estimates a face direction corresponding to the calculated face angle if the difference between the calculated face angle and the reference face angle is greater than or equal to the threshold and selects a sub image corresponding to the estimated face direction in a face area of the main image on the video call screen. 
     
     
         7 . The apparatus of  claim 6 , wherein the screen configurer reconfigures the video call screen by switching between the screen of the main image and the screen of the selected sub image on the video call screen. 
     
     
         8 . A method for configuring a screen for a video call using a facial expression, comprising:
 configuring a video call screen including multiple video images received for the video call;   recognizing a face from an image, and calculating facial expression information for an expression of the recognized face;   determining whether there is a change in expression of the recognized face by comparing the calculated facial expression information with reference expression information preset to determine a change in expression of the face;   selecting a video image corresponding to the changed expression in the video call screen if there is a change in expression; and   reconfiguring the video call screen using the selected video image.   
     
     
         9 . The method of  claim 8 , wherein the video call screen includes a screen of a main image received from a camera and a screen of at least one sub image received outside of the video call. 
     
     
         10 . The method of  claim 9 , wherein calculating facial expression information further comprises:
 recognizing a face in the main image;   extracting facial features of the recognized face; and   calculating a face angle of the recognized face based on the extracted facial features.   
     
     
         11 . The method of  claim 10 , further comprising calculating a face angle from an image received before the video call, and setting the calculated face angle as a reference face angle. 
     
     
         12 . The method of  claim 11 , wherein determining whether there is a change in expression of the recognized face further comprises determining whether a difference between the calculated face angle and the reference face angle is greater than or equal to a preset threshold by comparing the calculated face angle with the reference face angle. 
     
     
         13 . The method of  claim 12 , wherein selecting a video image corresponding to the changed expression comprises:
 estimating a face direction corresponding to the calculated face angle if the difference between the calculated face angle and the reference face angle is the threshold value or above; and   selecting a sub image corresponding to the estimated face direction in a face area of the main image on the video call screen.   
     
     
         14 . The method of  claim 13 , wherein reconfiguring the video call screen comprises switching between the screen of the main image and the screen of the selected sub image on the video call screen.