Online advertising marketplace data provider assessment and recommendation

Abstract

The present invention provides techniques for use in assessing the value of information-related services of particular data providers to an online advertising marketplace participant, such as an advertiser, publisher, market-maker, or another data provider, in connection with activities relating to buying, selling or pricing of marketplace properties. Furthermore, a recommendation may be provided to the marketplace participant as to the value or desirability of the services of particular data providers. Techniques are provided in which actual or hypothetical impact of use of information of the particular data providers is assessed.

1 . A method for use in association with an online advertising marketplace, comprising:
 using one or more computers, obtaining a first set of information comprising information of one or more data providers in connection with marketplace properties and for use by a marketplace participant in increasing effectiveness of activities relating to buying, selling, or pricing in connection with advertising marketplace properties;   using one or more computers, performing an assessment of an actual or hypothetical impact that use of the first set of information in connection with the activities had or may have had on effectiveness of the activities; and   using one or more computers, based at least in part on the assessed impact, determining a second set of information comprising information relating to a value, degree of desirability, or degree of appropriateness, of services of the one or more data providers in providing information to the marketplace participant.   
     
     
         2 . The method of  claim 1 , wherein the one or more data providers provide markup information relating to marketplace properties. 
     
     
         3 . The method of  claim 1 , wherein the one or more data providers provide markup information relating to marketplace properties, and wherein the marketplace is an auction-based marketplace including advertising bidding. 
     
     
         4 . The method of  claim 1 , wherein the one or more data providers provide markup information relating to marketplace properties, and wherein the marketplace properties comprise advertising or publishing inventory. 
     
     
         5 . The method of  claim 1 , comprising providing information of the second set of information for use by the marketplace participant, and wherein in the marketplace participant is an advertiser. 
     
     
         6 . The method of  claim 1 , comprising providing information of the second set of information for use by the marketplace participant, and wherein the marketplace participant is a publisher. 
     
     
         7 . The method of  claim 1 , comprising providing information of the second set of information for use by the marketplace participant, and wherein the marketplace participant is a market-maker or a marketplace facilitator. 
     
     
         8 . The method of  claim 1 , comprising providing information of the second set of information for use by a marketplace participant, and wherein the marketplace participant is a data provider. 
     
     
         9 . The method of  claim 1 , wherein performing the assessment comprises performing an assessment of a hypothetical impact that use of the first set of information may have had. 
     
     
         10 . The method of  claim 1 , wherein performing the assessment comprises performing an assessment of a hypothetical impact that use of the first set of information may have had, comprising comparing (a) hypothetical performance of a set of advertisement calls that would have been affected by markups of the first set of information with (b) actual performance of a set of advertisement calls including the advertisement calls that would have been affected by markups of the first set of information and advertisement calls that would not have been affected by markups of the first set of information. 
     
     
         11 . The method of  claim 1 , wherein performing the assessment comprises performing an assessment of an actual impact of use of the first set of information, comprising using one or more controlled experiments. 
     
     
         12 . The method of  claim 1 , wherein performing the assessment comprises performing an assessment of an actual impact of use of the first set of information, comprising using one or more controlled experiments, wherein the one or more controlled experiments include comparing (a) performance of a set of advertisement calls to which markup information of the first set of information is allowed to be applied with (b) performance of a set of advertisement calls to which markup information of the first set of information is not allowed to be applied. 
     
     
         13 . The method of  claim 1 , comprising using the second set of information in providing information to the marketplace participant relating to value or desirability of services of the one or more data providers to the marketplace participant. 
     
     
         14 . The method of  claim 1 , comprising utilizing the second set of information in providing a recommendation to the marketplace participant relating to whether the marketplace participant should use or should consider using services of the data provider. 
     
     
         15 . The method of  claim 1 , wherein assessing data providers includes use of one or more machine learning, classification, or regression-based techniques. 
     
     
         16 . The method of  claim 1 , wherein assessing data providers includes use of one or more recommender system techniques or collaborative filtering techniques. 
     
     
         17 . A system for use in association with an online advertising marketplace, comprising:
 one or more server computers coupled to a network; and   one or more databases coupled to the one or more server computers;   wherein the one or more server computers are for:
 obtaining, and storing in at least one of the one or more databases, a first set of information comprising information of one or more data providers in connection with marketplace properties and for use by a marketplace participant in increasing effectiveness of activities relating to buying, selling, or pricing in connection with advertising marketplace properties; 
 performing an assessment of an actual or hypothetical impact that use of the first set of information in connection with the activities had or may have had on effectiveness of the activities; and 
 based a least in part on the assessed impact, determining a second set of information comprising information relating to a value, degree of desirability, or degree of appropriateness, of services of the one or more data providers in providing information to the marketplace participant. 
   
     
     
         18 . The system of  claim 17 , comprising utilizing the second set of information in providing a recommendation to the marketplace participant relating to value or desirability of services of the data provider to the marketplace participant. 
     
     
         19 . The system of  claim 17 , wherein performing the assessment comprises performing an assessment of an actual impact of use of the first set of information, comprising using one or more controlled experiments, wherein the one or more controlled experiments include comparing (a) performance of a set of advertisement calls to which markup information of the first set of information is allowed to be applied with (b) performance of a set of advertisement calls to which markup information of the first set of information is not allowed to be applied. 
     
     
         20 . A computer readable medium or media containing instructions for executing a method for use in association with an online advertising marketplace comprising:
 using one or more computers, obtaining a first set of information comprising information of one or more data providers in connection with marketplace properties and for use by a marketplace participant in increasing effectiveness of activities relating to buying, selling, or pricing in connection with advertising marketplace properties;   using one or more computers, performing an assessment of an actual impact that use of the first set of information in connection with the activities had on effectiveness of the activities, comprising using one or more controlled experiments, wherein the one or more controlled experiments include comparing (a) performance of a set of advertisement calls to which markup information of the first set of information is allowed to be applied with (b) performance of a set of advertisement calls to which markup information of the first set of information is not allowed to be applied;   using one or more computers, based at least in part on the assessed impact, determining a second set of information comprising information relating to a value, degree of desirability, or degree of appropriateness, of services of the one or more data providers in providing information to the marketplace participant; and   using one or more computers, based at least in part on the second set of information, providing a third set of information to the marketplace participant relating to value or desirability of services of the data provider to the marketplace participant.