Segmenting Human Hairs and Faces

Abstract

Systems for segmenting human hairs and faces in color images are disclosed, with methods and processes for making and using the same. The image may be cropped around the face area and roughly centered. Optionally, the illumination environment of the input image may be determined. If the image is taken under dark environment or the contrast between the face and hair regions and background is low, an extra image enhancement may be applied. Sub-processes for identifying the pose angle and chin contours may be performed. A preliminary mask for the face by using multiple cues, such as skin color, pose angle, face shape and contour information can be represented. An initial hair mask by using the abovementioned multiple cues plus texture and hair shape information may be created. The preliminary face and hair masks are globally refined using multiple techniques.

1 - 3 . (canceled) 
     
     
         4 . A method comprising:
 generating a generic facial mask associated with a facial feature of a person, wherein the generic facial mask is generated based on a first plurality of training images, wherein the generic facial mask is configured to identify the facial feature of a particular person;   generating a generic hair mask associated with a hair feature of a person, wherein the generic hair mask is generated based on a second plurality of training images, wherein the generic hair mask is configured to identify the hair feature of the particular person; and   storing the generic hair and facial masks.   
     
     
         5 . The method of  claim 4 ,
 wherein the generic facial mask comprises a first plurality of control data points, wherein the first plurality of control data points is based on a first plurality of training data point sets, wherein each training data point set of the first plurality of training data point sets is associated with a facial feature of a training image of the first plurality of training images, and   wherein the generic hair mask comprises a second plurality of control data points, wherein the second plurality of control data points is based on a second plurality of training data point sets, wherein each training data point set of the second plurality of training data point sets is associated with a hair feature of a training image of the second plurality of training images.   
     
     
         6 . The method of  claim 4 , wherein the generic facial mask comprises a first plurality of control data points, and wherein the generating of the generic facial mask comprises averaging training data point sets associated with the first plurality of training images. 
     
     
         7 . The method of  claim 4 , wherein the generic hair mask comprises a second plurality of control data points, and wherein the generating of the generic hair mask comprises averaging training data point sets associated with the second plurality of training images. 
     
     
         8 . The method of  claim 4 , wherein the generic facial mask is a mask associated with a chin of a person. 
     
     
         9 . The method of  claim 4  further comprising:
 generating a skin model associated with a skin color of a person based on skin color intensity of a third plurality of training images; and 
 storing the skin model. 
 
     
     
         10 . A method comprising:
 receiving a desired image;   receiving a generic facial mask associated with a facial feature of a person from a memory component, wherein the generic facial mask is based on a first plurality of training images, and wherein the generic facial mask is configured to identify a facial feature of a person in an image;   receiving a generic hair mask associated with a hair feature of a person from the memory component, wherein the generic hair mask is based on a second plurality of training images, and wherein the generic hair mask is configured to identify a hair feature of a person in an image;   applying the generic facial mask and the generic hair mask to the desired image; and   identifying facial feature and hair feature in the desired image.   
     
     
         11 . The method of  claim 10 ,
 wherein the generic facial mask comprises a first plurality of control data points, wherein the first plurality of control data points is based on a first plurality of training data point sets, wherein each training data point set of the first plurality of training data point sets is associated with a facial feature of a training image of the first plurality of training images; and   wherein the generic hair mask comprises a second plurality of control data points, wherein the second plurality of control data points is based on a second plurality of training data point sets, wherein each training data point set of the second plurality of training data point sets is associated with a hair feature of a training image of the second plurality of training images.   
     
     
         12 . The method of  claim 11 , wherein applying the generic facial mask and the generic hair mask comprises iteratively applying the first plurality of control data points to the desired image to identify the facial feature in the desired image and iteratively applying the second plurality of control data points to the desired image to identify the hair feature in the desired image. 
     
     
         13 . The method of  claim 11 , wherein the identified facial feature is substantially a match of the first plurality of control data points in the desired image, and wherein the identified hair feature is substantially a match of the second plurality of control data points in the desired image. 
     
     
         14 . The method of  claim 10  further comprising:
 refining the identified hair feature and the identified facial feature in the desired image; and 
 forming a head mask of a person in the desired image based on the refined hair and facial features. 
 
     
     
         15 . The method of  claim 10  further comprising:
 dividing the identified hair feature into a plurality of hair regions, wherein each hair region of the plurality of hair regions has a hair color associated therewith; 
 classifying each hair region of the plurality of hair regions by a hair color, 
 determining an overall hair color associated with the identified hair feature based on a majority of the hair classification; 
 identifying a portion in the desired image that includes the overall hair color; and 
 refining the identified hair feature by including the identified portion into the identified hair feature. 
 
     
     
         16 . The method of  claim 10  further comprising:
 applying a plurality of hair contour data points to the desired image to identify an outer hair boundary between the identified hair feature and a background portion of the desired image; 
 identifying the outer hair boundary in response the outer hair boundary substantially matching the plurality of hair contour data points; 
 identifying uneven portions at the outer hair boundary; 
 smoothing out the uneven portions to form a new outer hair boundary that is substantially convex contour shaped; and 
 updating the identified hair features to include the new outer hair boundary. 
 
     
     
         17 . The method of  claim 10  further comprising:
 determining an illumination level based on an average illumination level of the identified facial feature; 
 determining whether the illumination level exceeds an illumination level threshold; and 
 enhancing image quality of the identified facial feature in response to the illumination level exceeding the illumination level threshold. 
 
     
     
         18 . The method of  claim 10 , wherein the generic facial mask is associated with a chin feature of a person, and wherein the method further comprises:
 identifying a chin contour in the desired image in response to a match resulting from application of the generic facial mask to the desired image;   determining whether the identified facial feature includes portions outside of the chin contour;   refining the identified facial feature by removing the determined portions from the identified facial feature in response to determining that the identified facial feature includes portions outside of the chin contour, and   responsive to the refining, smoothing uneven portions along the chin contour to form a new chin contour, wherein the new chin contour is substantially convex shaped.   
     
     
         19 . The method of  claim 10 , wherein applying the generic hair mask includes applying a bald head template to the desired image, and wherein the method further comprises:
 determining that a person in the desired image has a bald head if a match between the facial feature and the bald head template is within a predetermined threshold; and   responsive to determining that a person in the desired image is bald, refining the identified hair features by removing portions from the identified hair features that are outside of a boundary represented by the bald head template.   
     
     
         20 . A method comprising:
 generating a generic feature mask associated with a facial feature of a person that is substantially invariant from one person to another person, wherein the generic feature mask is based on a plurality of training images; and   storing the generic feature mask.   
     
     
         21 . The method of  claim 20 , wherein the generic feature mask comprises a plurality of control data points, wherein the plurality of control data points is based on a plurality of training data point sets, wherein each training data point set of the plurality of training data point sets is associated with a facial feature of a training image of the plurality of training images. 
     
     
         22 . The method of  claim 20 , wherein the generic feature mask comprises a plurality of control data points, and wherein the generating the generic feature mask comprises averaging training data point sets associated with the plurality of training images. 
     
     
         23 . The method of  claim 20 , wherein the facial feature of the person is selected from a group consisting of a face, a nose, eyes, a mouth, and a chin.