Recommendation system capable of adapting to user feedback

Abstract

A recommendation system uses feedback from users on specific item recommendations to assess the quality of the recommendation rules used to generate such recommendations. The feedback may be explicit (e.g., a user rates a particular recommended item), implicit (e.g., a user purchases a recommended item), or both. The system may use these assessments to modify the degree to which particular recommendation rules are used to generate recommendations. For instance, if a particular recommendation rule leads to negative feedback relatively frequently, the system reduce or terminate its reliance on the rule. In some embodiments, the system may also increase its reliance on recommendation rules that tend to produce positive feedback.

1 . A recommendation system, comprising:
 a first computer data repository of recommendation rules;   a recommendation engine configured to use the recommendation rules, in combination with data regarding item selections of users, to select items to recommend to the users;   a server configured to provide a recommendation user interface for the users to view item recommendations generated with the recommendation engine, said recommendation user interface including functionality for users to provide explicit feedback on particular item recommendations;   a second computer data repository that records the explicit feedback in association with the recommendation rules to which such feedback corresponds; and   a feedback-based adjuster that adjusts personalized recommendation sets generated by the recommendation engine based on the recorded feedback.   
     
     
         2 . The recommendation system of  claim 1 , wherein the second computer data repository stores, for a first recommendation rule, statistical data regarding the feedback provided by users on item recommendations generated with the first recommendation rule. 
     
     
         3 . The recommendation system of  claim 2 , wherein the feedback-based adjuster is configured to use the statistical data to determine whether to modify a ranking of an item recommendation generated with the first recommendation rule. 
     
     
         4 . A method for adaptively using recommendation rules to generate recommendations of items, the method comprising:
 identifying a first item selected by a user;   selecting, by a recommendation service, a second item to recommend to the user based at least partly on (a) the user's selection of the first item, and (b) a recommendation rule that associates the first item with the second item;   outputting a recommendation of the second item to the user via a user interface that provides an option for the user to provide explicit feedback on the recommendation;   recording, in association with the recommendation rule, the user's explicit feedback on the recommendation of the second item;   generating a score that represents a level at which the recommendation rule has performed, said score based on the explicit feedback provided by the user, and based additionally explicit feedback provided by other users on other recommendations generated using the recommendation rule; and   based at least partly on the score, modifying the recommendation service's use of the recommendation rule to provide recommendations to users;   said method performed by a computing system that comprises one or more computing devices.   
     
     
         5 . The method of  claim 4 , wherein the user interface is a personalized recommendations page that displays a plurality of items selected by the recommendation service to recommend to the user, said personalized recommendations page providing an option for the user to explicitly rate each of the plurality of items. 
     
     
         6 . The method of  claim 4 , wherein the explicit feedback is a rating assigned to the second item by the user via a personalized recommendations page. 
     
     
         7 . The method of  claim 4 , wherein outputting the recommendation of the second item comprises outputting to the user an indication that the recommendation of the second item is based on the user's selection of the first item. 
     
     
         8 . The method of  claim 4 , wherein modifying the recommendation service's use of the recommendation rule comprises, based on the score, adjusting a display rank of the second item within a set of personalized item recommendations generated for the user. 
     
     
         9 . The method of  claim 4 , wherein modifying the recommendation service's use of the recommendation rule comprises inhibiting the recommendation rule from being used to recommend the second item to users. 
     
     
         10 . The method of  claim 4 , wherein the score is based additionally on implicit feedback provided by users on recommendations generated by the recommendations service with the recommendation rule. 
     
     
         11 . The method of  claim 4 , wherein the recommendation rule includes a weight value representing a strength of an association between the first and second items, said weight value being distinct from the score. 
     
     
         12 . The method of  claim 4 , wherein the recommendation is based additionally on a second recommendation rule that associates a third item with the second item, and the method further comprises recording the user's explicit feedback on the recommendation in association with the second recommendation rule. 
     
     
         13 . The method of  claim 4 , wherein recording the explicit feedback in association with the recommendation rule comprises generating a record specifying that the explicit feedback resulted from an item recommendation generated with the recommendation rule. 
     
     
         14 . Non-transitory computer storage that stores executable program instructions that direct a computing system to perform a process that comprises:
 identifying a first item selected by a user;   selecting a second item to recommend to the user based at least partly on (a) the user's selection of the first item, and (b) a recommendation rule that associates the first item with the second item;   outputting a recommendation of the second item to the user via a user interface that provides an option for the user to provide explicit feedback on the recommendation;   recording, in association with the recommendation rule, the user's explicit feedback on the recommendation of the second item;   generating a score that represents a level at which the recommendation rule has performed, said score based on the explicit feedback provided by the user, and based additionally on explicit feedback provided by other users on other recommendations generated using the recommendation rule; and   based at least partly on the score, modifying use of the recommendation rule to provide recommendations to users.   
     
     
         15 . The non-transitory computer storage of  claim 14 , wherein modifying use of the recommendation rule comprises adjusting, based on the score, a rank of the second item within a set of personalized item recommendations. 
     
     
         16 . The non-transitory computer storage of  claim 14 , in combination with the computing system, wherein the computing system is programmed with the executable program instructions to perform the process. 
     
     
         17 . A method of providing item recommendations that depend on collective user feedback, the method comprising:
 receiving a personalized recommendation set from a recommendation engine, said personalized recommendation set specifying a plurality of items selected to recommend to a user, and specifying a ranking of said items, said recommendation set generated by the recommendation engine based on item preference information of the user and based additionally on recommendation rules;   identifying, in connection with the personalized recommendation set, a recommendation rule used by the recommendation engine to select a first item of said plurality of items;   determining a score for the recommendation rule, said score representing a level at which the recommendation rule has performed as determined based on feedback provided by users on item recommendations generated with the recommendation rule; and   adjusting a ranking of the first item in the personalized recommendation set based at least partly on the score;   said method performed by a computing system that comprises one or more computing devices.   
     
     
         18 . The method of  claim 17 , wherein the feedback comprises ratings explicitly assigned by users to particular recommended items via a recommendation interface. 
     
     
         19 . The method of  claim 17 , wherein the recommendation rule comprises an item-to-item association mapping. 
     
     
         20 . Non-transitory computer storage that stores executable program instructions that direct a computing system to perform a process that comprises:
 receiving a personalized recommendation set from a recommendation engine, said personalized recommendation set specifying a plurality of items selected to recommend to a user, and specifying a ranking of said items, said recommendation set generated by the recommendation engine based, at least in part, on item preference information of the user and recommendation rules that specify associations between particular items;   identifying a recommendation rule used by the recommendation engine to select a first item of said plurality of items;   determining a score for the recommendation rule, said score representing a performance level of the recommendation rule based on feedback provided by users on item recommendations generated with the recommendation rule; and   adjusting a ranking of the first item in the personalized recommendation set based at least partly on the score.   
     
     
         21 . The non-transitory computer storage of  claim 20 , wherein the feedback comprises ratings explicitly assigned by the users to particular recommended items via a recommendation interface. 
     
     
         22 . The non-transitory computer storage of  claim 20 , in combination with the computing system, wherein the computing system is programmed with the executable program instructions to perform the process.