Studying aesthetics in photographic images using a computational approach

Abstract

The aesthetic quality of a picture is automatically inferred using visual content as a machine learning problem using, for example, a peer-rated, on-line photo sharing Website as data source. Certain visual features of images are extracted based on the intuition that they can discriminate between aesthetically pleasing and displeasing images. A one-dimensional support vector machine is used to identify features that have noticeable correlation with the community-based aesthetics ratings. Automated classifiers are constructed using the support vector machines and classification trees, with a simple feature selection heuristic being applied to eliminate irrelevant features. Linear regression on polynomial terms of the features is also applied to infer numerical aesthetics ratings.

1 . A computer-based method of inferring and utilizing the aesthetic quality of photographs and other images, comprising the steps of:
 providing a collection of previously digitized images and a digitized image to be evaluated;   performing one or more software operations on the digitized image to automatically extract a plurality of visual features representative of the image; and   using one or more of the visual features to classify the image on the basis of its aesthetic value, rate the image on a scale relating to its aesthetics value, or select/eliminate the image as being aesthetically distinctive when compared to the collection.   
   
   
       2 . The method of  claim 1 , including the steps of:
 receiving a plurality of digitized images along with aesthetic-based ratings of the images provided by viewers of the images;   performing one or more software operations on the images to identify those features which correlate to the aesthetic-based ratings provided by the viewers; conceiving, testing, and isolating the features which correlate with these ratings, in order to classify the image, automatically rate the image, select or eliminate the image from a collection, all based on its perceived aesthetic value.   
   
   
       3 . The method of  claim 2 , wherein the correlated visual features are identified using a one-dimensional and multi-dimensional support vector machines. 
   
   
       4 . The method of  claim 1 , further including the step of converting the image into the HSV color space to produce two-dimensional matrices IH, IS and IV. 
   
   
       5 . The method of  claim 1 , further including the step of segmenting the object to identify objects in the image. 
   
   
       6 . The method of  claim 1 , wherein the feature is exposure. 
   
   
       7 . The method of  claim 1 , wherein the feature is color distribution. 
   
   
       8 . The method of  claim 1 , wherein the feature is saturation or hue. 
   
   
       9 . The method of  claim 1 , wherein the feature is based upon the rule of thirds. 
   
   
       10 . The method of  claim 1 , wherein the feature is based upon viewer familiarity or identifiability with the image composition, relating to its originality. 
   
   
       11 . The method of  claim 1 , wherein the feature is graininess or smoothness. 
   
   
       12 . The method of  claim 1 , wherein the feature is image size or aspect ratio. 
   
   
       13 . The method of  claim 1 , wherein the feature is photographic depth-of-field. 
   
   
       14 . The method of  claim 1 , wherein the feature is related to shape regularity of segments within the image. 
   
   
       15 . The method of  claim 1 , further including the step of performing a linear regression on one or more of the features to infer the aesthetics of the image. 
   
   
       16 . The method of  claim 1 , further including the use of a naive Bayes classifier to classify the image based on its aesthetic value. 
   
   
       17 . The method of  claim 1 , wherein linear regression and naive Bayes classification methods are used in conjunction to select high-quality images or eliminate low-quality images from image collections, on the basis of their aesthetic values. 
   
   
       18 . The method of  claim 1 , wherein the step of using one or more of the visual features includes the employment of a content-based image retrieval algorithm. 
   
   
       19 . The method of  claim 1 , wherein the step of using one or more of the visual features includes a composition suggestion within a digital photography system for enhancing the aesthetic value of capture images. 
   
   
       20 . The method of  claim 1 , wherein the step of using one or more of the visual features includes discriminating between visually similar images.