Real-time adaptive probabilistic selection of messages

Abstract

A facility selects a unit of content to include in an instance of a content container. For each of a plurality of units of content, across a plurality of other instances of the content container distributed during a foregoing period of time that included the unit of content, the facility determines both an average level of success achieved by including the unit of content, and a measure of the level of variability of the level of success achieved by including the unit of content. The facility scores each unit of content by selecting a value from a probability distribution that is based on the determined average level of success and measure of the level of variability of the level of success. The facility selects one of the units of content to include in the instance of the content container based on the determined scores.

1 . A method comprising:
 determining, across a plurality of instances of a web page served during a foregoing period of time that included an advertising message,
 (a) a set of values attributable to inclusion of the advertising message in each of the plurality of instances of the web page, 
 (b) an average value of the set of values attributable to the inclusion of the advertising message in the plurality of instances of the web page, and 
 (c) a measure of a level of variability of the set of values attributable to inclusion of the advertising message in the plurality of instances of the web page; and 
   determining a score for the advertising message reflective of the average value and the measure of the level of variability of the values attributable to the inclusion of the advertising message in the plurality of instances of web pages,
 wherein determining the score for the advertising message includes performing a weighted random selection of a value from a normal probability distribution centered on the determined average value, and 
 wherein a breadth of the normal probability distribution is based on the determined measure of the level of variability of the set of values, and 
   
       wherein the method is implemented in a computing system having a processor and a memory. 
     
     
         2 . The method of  claim 1 , wherein determining the score for the advertising message is further reflective of an exposure guarantee associated with the advertising message. 
     
     
         3 . The method of  claim 2 , wherein determining the measure of the level of variability of the set of values attributable to the inclusion of the advertising message used as a basis for determining the score for the advertising message with the associated exposure guarantee comprises determining the measure of the level of variability of the set of values attributable to the inclusion of the advertising message augmented by a guarantee bonus value. 
     
     
         4 . The method of  claim 1 , further comprising: determining that the advertising message has been included in the plurality of instances of the web page in less than a threshold number of times during the foregoing period of time. 
     
     
         5 . The method of  claim 1 , wherein determining the average value of the set of values comprises determining a constructive average value based on an aggregate average value of all advertising messages included in the plurality of instances of the web page during a foregoing period of time. 
     
     
         6 . The method of  claim 1 , wherein determining the measure of the level of variability of the set of values comprises determining a constructive measure of the level of variability of the set of values associated with the inclusion of all messages included in the plurality of instances of the web page during a foregoing period of time. 
     
     
         7 . The method of  claim 1 , further comprising:
 identifying a plurality of candidate advertising messages eligible to be included in the web page during a subsequent period of time;   determining the score for each of the plurality of candidate advertising messages;   selecting one of the plurality of candidate advertising messages as an identified advertising message to include in the web page during the subsequent period of time, wherein the selecting is based on the score of the respective candidate advertising messages.   
     
     
         8 . The method of  claim 7 , wherein the selecting based on the score comprises selecting the candidate advertising message with the highest score. 
     
     
         9 . The method of  claim 7 , wherein determining the score for each of the plurality of the candidate advertising messages includes determining the exposure guarantee and wherein selecting the identified candidate message includes selecting only a candidate message associated with an exposure guarantee. 
     
     
         10 . A system for managing advertising messages, comprising:
 a computing system having a processor and memory for implementing an analysis subsystem that determines, for an advertising message, across a plurality of instances of a web page distributed during a foregoing period of time that included the advertising message (a) a measure of success achieved by including the advertising message in one instance of the web page (b) an average of the measure of success achieved by including the advertising message across the plurality of instances of the web page, and (c) a measure of the level of variability of the measure of success achieved by including the advertising message; and   a computing system having a processor and memory for implementing a scoring subsystem that determines a score for the advertising message by performing a weighted random selection of a value from a probability distribution based on (a) the determined average measure of success and (b) the determined measure of the level of variability of the measure of success.   
     
     
         11 . The system of  claim 10  further comprising a computing system having a processor and memory for implementing a selection subsystem that selects at least one of a plurality of advertising messages to include in a subsequent instance of the web page based on the scores determined for each of the plurality of advertising messages. 
     
     
         12 . The system of  claim 11 , wherein the selection subsystem selects the at least one of the plurality of advertising messages to include in the subsequent instance of the web page further based on application of a suitability test associated with the advertising message. 
     
     
         13 . The system of  claim 10 , wherein the analysis subsystem monitors (a) the determined average of the measure of success and (b) the determined measure of the level of variability of the measure of success for the advertising message over time to take into account the inclusion of the advertising message in the subsequent instance of the web page and associated measure of success. 
     
     
         14 . The system of  claim 10 , wherein an exposure guarantee is associated with the advertising message. 
     
     
         15 . The system of  claim 11 , wherein the plurality of advertising messages are subject to respective exposure guarantees, and wherein, at a time when at least one of the respective exposure guarantees is not satisfied, the selection of an advertising message not subject to an exposure guarantee is precluded. 
     
     
         16 . A computer-readable, non-transitory storage medium comprising:
 one or more computer-executable components for managing advertising messages, the one or more computer-executable components:   determining, across a plurality of instances of a web page served during a previous period of time that included an advertising message,
 (a) a set of success rates attributable to inclusion of the advertising message in each of the plurality of instances of the web page, 
 (b) an average of the success rates attributable to the inclusion of the advertising message in the plurality of instances of the web page, and 
 (c) a measure of a standard error of the success rates associated with the inclusion of the advertising message; and 
   determining a score for the advertising message reflective of the average of the success rates and the standard error of the success rates attributable to the inclusion of the advertising message in the plurality of instances of web pages,
 wherein determining the score for the advertising message includes performing a weighted random selection of a success rates from a normal probability distribution centered on the determined average success rate, 
 wherein the breadth of the normal probability distribution is based on the determined measure of the standard error of the success rates. 
   
     
     
         17 . The computer-readable medium of  claim 16 , the one or more computer-executable components further:
 identifying a plurality of candidate advertising messages eligible to be included in the web page during a subsequent period of time; and   determining the score for each of the plurality of candidate advertising messages;   selecting one of the plurality of candidate advertising messages as an identified advertising message to include in the web page during the subsequent period of time, wherein the selecting is based on the score of the respective candidate advertising messages.   
     
     
         18 . The computer-readable medium of  claim 17 , wherein the selecting based on the score comprises selecting the candidate advertising message with the highest score. 
     
     
         19 . The computer-readable medium of  claim 16 , wherein an exposure guarantee is associated with the advertising message. 
     
     
         20 . The computer-readable medium of  claim 19 , wherein the determined measure of the standard error of the success rates associated with the inclusion of the advertising message used as a basis for determining the score for the advertising message with the associated exposure guarantee comprises the determined measure of the standard error of the success rates associated with the inclusion of the advertising message augmented by a guarantee bonus success rate. 
     
     
         21 . The computer-readable medium of  claim 19 , wherein a plurality of the identified candidate messages are each the subject of an exposure guarantee, and wherein, at a time when at least one of the exposure guarantees is not satisfied, the selection of any identified candidate messages that are not the subject of an exposure guarantee is precluded. 
     
     
         22 . The computer-readable medium of  claim 19 , wherein the advertising message has been included in the plurality of instances of the web page in less than a threshold number of times during the previous period of time. 
     
     
         23 . The computer-readable medium of  claim 19 , wherein the average of the success rates is a constructive average success rate based on an aggregate average success rate of all advertising messages included in the plurality of instances of the web page during the previous period of time. 
     
     
         24 . The computer-readable medium of  claim 19 , wherein the measure of the standard error is a constructive measure of the standard error of the success rates associated with the inclusion of all messages included in the plurality of instances of the web page during a previous period of time.