Method and system for unsupervised image segmentation using a trained quality metric

Abstract

A method and apparatus for unsupervised segmentation of an image is provided. In some exemplary embodiments, the method adjusts a segmentation parameter of a traditional graph-based segmentation algorithm along the image to generate a segmentation map that is perceptually reasonable for a human observer. In some embodiments, the method reduces over-segmentation and under-segmentation of the image.

1 . A method of segmenting an image, comprising:
 providing an image;   determining a first value of a segment parameter, wherein the segment parameter relates to a threshold function for establishing a boundary condition between a first segment and a second segment;   determining a first value of a similarity function configured to indicate a similarity between the image and its segmentation based on the first value of the segment parameter;   comparing the first value of the segment parameter and the first value of the similarity function to a predetermined function;   determining a second value of the segment parameter based on a result of the comparing; and   segmenting the image based on the second value of the segment parameter.   
     
     
         2 . The method of  claim 1 , wherein the similarity function comprises a symmetric uncertainty function. 
     
     
         3 . The method of  claim 2 , wherein the predetermined function is a linear function representing a linear relationship between the log of the segment parameter and the symmetric uncertainty, wherein a value above the linear function indicates over-segmentation and a value below the linear function indicates under-segmentation. 
     
     
         4 . The method of  claim 3 , further comprising determining an optimal value of the segment parameter, the optimal value of the segment parameter comprising a value of the segment parameter that generates a segmentation of the image for which a difference between a corresponding value of the symmetric uncertainty and a portion of the linear relationship is minimized. 
     
     
         5 . The method of  claim 1 , wherein segmenting the image based on the second value of the segment parameter comprises:
 dividing the image into a plurality of sub-images, wherein the sub-images are overlapping or non-overlapping;   generating a scale map for the image by determining a plurality of values of the segment parameter, wherein each of the plurality of values corresponds to one of the plurality of sub-images; and   smoothing the scale map for the image using a filter.   
     
     
         6 . The method of  claim 5 , wherein the filter comprises a low-pass filter. 
     
     
         7 . The method of  claim 5 , further comprising:
 providing an additional image, wherein the additional image is disposed subsequent to the image in a video;   dividing the additional image into an additional plurality of sub-images, wherein the additional plurality of sub-images corresponds to the plurality of sub-images in at least one of size and location;   providing the plurality of values of the segment parameter as a plurality of initial estimates for the segment parameter corresponding to the additional plurality of sub-images;   determining a plurality of optimized values of the segment parameter, wherein each of the plurality of optimized values corresponds to one of the additional plurality of sub-images; and   segmenting the additional image based on the plurality of optimized values of the segment parameter.   
     
     
         8 . The method of  claim 2 , wherein determining the linear function includes:
 providing a plurality of training images;   generating a segmentation map for each of the plurality of training images at a plurality of values of the segment parameter;   determining a value of a symmetric uncertainty for each segmentation map; and   classifying each segmentation map as being over-segmented, well segmented, or under-segmented, based on a visual perception by at least one observer.   
     
     
         9 . A method of segmenting an image, comprising:
 providing an image;   dividing the image into a plurality of sub-images, each sub-image comprising a plurality of pixels; and for each sub-image, the method comprising:
 determining a first value of a parameter, wherein the parameter relates to a threshold function for establishing a boundary condition between a first segment and a second segment; 
 determining a first value of a symmetric uncertainty of the sub-image based on the first value of the first parameter; 
 comparing the first value of the parameter and the first value of the symmetric uncertainty to a predetermined function; and 
 determining a second value of the parameter based on a result of the comparing. 
   
     
     
         10 . The method of  claim 9 , further comprising:
 assigning the determined value of the parameter to each pixel in the sub-image;   applying a filter to the assigned values to obtain a filtered value of the parameter for each pixel in the sub-image; and   segmenting the image based on the filtered values of the parameter.   
     
     
         11 . The method of  claim 10 , wherein the filter comprises a low-pass filter. 
     
     
         12 . The method of  claim 9 , further comprising:
 providing an additional image;   dividing the additional image into an additional plurality of sub-images;   segmenting the additional image based in part on the second value of the parameter determined for each of the first plurality of sub-images of the image.   
     
     
         13 . The method of  claim 9 , wherein the predetermined function is a linear function representing a linear relationship between the log of the first parameter and the symmetric uncertainty, wherein a value above the linear function indicates over-segmentation and a value below the linear function indicates under-segmentation. 
     
     
         14 . The method of  claim 13 , wherein determining the linear function includes:
 providing a plurality of training images;   generating a segmentation map for each of the plurality of training images at a plurality of values of the segment parameter;   determining a value of a symmetric uncertainty for each segmentation map; and   classifying each segmentation map as being over-segmented, well segmented, or under-segmented, based on a visual perception by at least one observer.   
     
     
         15 . A system, comprising:
 an image segmentation device, the image segmentation device comprising a processor and a memory, the memory comprising computer-readable media having computer-executable instructions embodied thereon that, when executed by the processor, cause the processor to instantiate one or more components, the one or more components comprising:
 a segment module configured to (1) determine a functional relationship between a first parameter and a second parameter based on input electronically received about a plurality of training images; and (2) for a sub-image of an image to be segmented:
 determine an initial value of the first parameter for the sub-image; and 
 determine an initial value of the second parameter; and 
 
 a comparison module configured to perform a comparison between the initial value of the first parameter and the initial value of the second parameter to the functional relationship; 
 wherein the segment module is further configured to determine an updated value of the first parameter based on the comparison. 
   
     
     
         16 . The system of  claim 15 , wherein the segment module is further configured to segment the image, based in part on the updated value of the first parameter, to create segmented image data. 
     
     
         17 . The system of  claim 16 , further comprising an encoder configured to encode the segmented image data. 
     
     
         18 . The system of  claim 17 , further comprising a communication module configured to facilitate communication of at least one of the image to be segmented and the segmented image data. 
     
     
         19 . The system of  claim 17 , wherein the segment module is further configured to:
 divide the additional image into a plurality of sub-images; and   segment the additional image, based in part on the updated value of the first parameter.   
     
     
         20 . The system of  claim 15 , wherein the second parameter comprises a symmetric uncertainty of the sub-image.