On-site composition and aesthetics feedback through exemplars for photographers

Abstract

A comprehensive system to enhance the aesthetic quality of the photographs captured by mobile consumers provides on-site composition and aesthetics feedback through retrieved examples. Composition feedback is qualitative in nature and responds by retrieving highly aesthetic exemplar images from the corpus which are similar in content and composition to the snapshot. Color combination feedback provides confidence on the snapshot to contain good color combinations. Overall aesthetics feedback predicts the aesthetic ratings for both color and monochromatic images. An algorithm is used to provide ratings for color images, while new features and a new model are developed to treat monochromatic images. This system was designed keeping the next generation photography needs in mind and is the first of its kind. The feedback rendered is guiding and intuitive in nature. It is computed in situ while requiring minimal input from the user.

The invention claimed is: 
     
       1. A method of providing aesthetic feedback to photographers, comprising the steps of:
 receiving a digital photograph from an image-capturing device at a server; 
 analyzing attributes of the photograph at the server, including the step of forming color triplets associated with the photograph; 
 providing feedback to a user of the image-capturing device regarding the aesthetic quality of the photograph; and 
 wherein the aesthetic quality is based at least in part on the color triplets. 
 
     
     
       2. The method of  claim 1 , including the steps of:
 storing a database of photographs at the server; 
 using the analyzed attributes of the photograph to determine if any of the photographs in the database are similar; and 
 providing feedback by retrieving and delivering any similar photographs to the image-capturing device. 
 
     
     
       3. The method of  claim 2 , wherein similar photographs are retrieved and delivered based on visual similarity, composition categorization or aesthetic scores. 
     
     
       4. The method of  claim 1 , including the steps of:
 analyzing attributes of the photograph at the server to determine an aesthetic rating associated with the image; and 
 providing the rating to the user as feedback. 
 
     
     
       5. The method of  claim 1 , wherein the step of analyzing attributes of the photograph includes analyzing the composition of the photograph. 
     
     
       6. The method of  claim 5 , wherein the step of analyzing the composition of the photograph includes determining if the photograph is textured or non-textured. 
     
     
       7. The method of  claim 5 , wherein the step of analyzing the composition of the photograph includes determining if the photograph includes a diagonal component. 
     
     
       8. The method of  claim 5 , wherein the step of analyzing the composition of the photograph includes determining if the photograph contains horizontal, centered, or vertical subject matter. 
     
     
       9. The method of  claim 1 , including the steps of:
 providing a database of photographs with predetermined aesthetic ratings; 
 analyzing low-level features of the photograph; 
 retrieving one or more photographs from the database having the highest aesthetic ratings based upon the low-level features; and 
 providing feedback to the user by recommending one or more of the photographs having the highest aesthetic ratings. 
 
     
     
       10. The method of  claim 9 , wherein:
 the photograph is a color photograph; and 
 the low-level features include one or more of light, colorfulness, size and aspect ratio. 
 
     
     
       11. The method of  claim 1 , wherein the feedback includes assisting a user with automatic composition analysis or color evaluation. 
     
     
       12. The method of  claim 1 , wherein the feedback includes the user's past photo-taking behavior. 
     
     
       13. The method of  claim 1 , wherein the image-capturing device and the server are integrated into a single, portable unit. 
     
     
       14. The method of  claim 1 , wherein:
 the image-capturing device is in network communication with the server; and 
 the digital photograph is sent to the server and the feedback is sent to the image-capturing device over the network. 
 
     
     
       15. The method of  claim 1 , wherein the image-capturing device is a mobile, wireless device. 
     
     
       16. A system for providing aesthetic feedback to photographers, comprising:
 a device for capturing a digital photograph; 
 a server for receiving the digital photograph; 
 a processor at the server operative to (a) analyze attributes of the photograph to form color triplets associated with the photograph, and (b) provide feedback to a user regarding the aesthetic quality of the photograph, with the feedback being based at least in part on the color triplets. 
 
     
     
       17. The system of  claim 16 , further including:
 a database at the server for storing photographs; and 
 wherein the processor is further operative to use the analyzed attributes of the photograph to determine if any of the photographs in the database are similar, and provide feedback by retrieving and delivering any similar photographs to the image-capturing device. 
 
     
     
       18. The system of  claim 17 , wherein similar photographs are retrieved and delivered based on visual similarity, composition categorization or aesthetic scores. 
     
     
       19. The system of  claim 16 , wherein the processor is further operative to analyze attributes of the photograph to determine an aesthetic rating associated with the image, and provide the rating to the user as feedback. 
     
     
       20. The system of  claim 16 , wherein the processor is further operative to analyze the composition of the photograph. 
     
     
       21. The system of  claim 20 , wherein the composition includes whether the photograph is textured or non-textured. 
     
     
       22. The system of  claim 20 , wherein the composition includes whether the photograph has a diagonal component. 
     
     
       23. The system of  claim 20 , wherein the composition includes whether the photograph has horizontal, centered, or vertical subject matter. 
     
     
       24. The system of  claim 16 , wherein:
 the server includes a database of photographs with predetermined aesthetic ratings; and 
 the processor is further operative to:
 analyze low-level features of the photograph; 
 retrieve one or more photographs from the database having the highest aesthetic ratings based upon the low-level features, and 
 provide feedback to the user by recommending one or more of the photographs having the highest aesthetic ratings. 
 
 
     
     
       25. The system of  claim 24 , wherein:
 the photograph is a color photograph; and 
 the low-level features include one or more of light, colorfulness, size and aspect ratio. 
 
     
     
       26. The system of  claim 16 , wherein the feedback includes assisting a user with automatic composition analysis or color evaluation. 
     
     
       27. The system of  claim 16 , wherein the feedback includes a user's past photo-taking behavior. 
     
     
       28. The system of  claim 16 , wherein the image-capturing device and the server are integrated into a single, portable unit. 
     
     
       29. The system of  claim 16 , wherein:
 the image-capturing device is in network communication with the server; and 
 the digital photograph is sent to the server and the feedback is sent to the image-capturing device over the network. 
 
     
     
       30. The system of  claim 16 , wherein the image-capturing device is a mobile, wireless device.