System and method for rating electronic documents

Abstract

A system and method for rating an electronic document such as an advertisement. Rating information is received from one or more evaluators. A signal relevant to a criteria is received and a determination is made whether to deliver the document in response to the signal based on the criteria and the rating information from the one or more evaluators.

1 . A computer-implemented method, comprising:
 receiving, at a server, rating information associated with contents of a document from one or more evaluators;   identifying, at the server, a trust score for each of the one or more evaluators, each trust score being a measure of an extent to which a respective evaluator's rating information is considered in determining an aggregate rating, each trust score based on factors associated with the respective evaluator;   identifying a sensitivity score, the sensitivity score indicating a level of conservatism in determining the aggregate rating;   determining the aggregate rating for the document based on the rating information, the trust scores, and the sensitivity score;   receiving a request from a recipient for one or more documents;   identifying a preferred aggregate rating to be associated with the request; and   determining whether to deliver the document in response to the request based on whether the aggregate rating satisfies the preferred aggregate rating.   
   
   
       2 . The method of  claim 1 , wherein identifying a preferred aggregate rating to be associated with the request comprises:
 receiving the preferred aggregate rating from the recipient.   
   
   
       3 . The method of  claim 1 , wherein the preferred aggregate rating is associated with a user or is stored in a profile. 
   
   
       4 . The method of  claim 1 , wherein each trust score is based on a geographical location of a respective evaluator. 
   
   
       5 . The method of  claim 1 , wherein each trust score is based on one or more prior content ratings received from the respective evaluator. 
   
   
       6 . The method of  claim 1 , wherein each trust score is based on an industry associated with the respective evaluator. 
   
   
       7 . The method of  claim 1 , further comprising:
 identifying a suitability standard associated with the recipient.   
   
   
       8 . The method of  claim 7 , wherein determining the aggregate rating for the document based on the rating information, the trust scores, and the sensitivity score comprises:
 determining the aggregate rating for the document based on the rating information, the trust scores, the sensitivity score, and the suitability standard.   
   
   
       9 . The method of  claim 1 , wherein the aggregate rating comprises a mean of the rating information. 
   
   
       10 . The method of  claim 1 , wherein the aggregate rating comprises a mode of the rating information. 
   
   
       11 . The method of  claim 1 , wherein the aggregate rating comprises a median of the rating information. 
   
   
       12 . A computer-implemented method, comprising:
 associating evaluation information with documents, the evaluation information comprising trust information and rating information;   receiving a request at a server for one or more of the documents;   ranking the documents based on the evaluation information in response to the request;   providing the ranked documents; and   updating the trust information based on the ranking.   
   
   
       13 . The method of  claim 12 , further comprising:
 receiving the rating information from one or more evaluators.   
   
   
       14 . The method of  claim 12 , wherein the trust information includes trust scores being a measure of an extent to which a respective evaluator's rating information is considered in determining an aggregate rating, each trust score based on factors associated with the respective evaluator. 
   
   
       15 . The method of  claim 14 , wherein each trust score is based on one or more prior content ratings received from the respective evaluator. 
   
   
       16 . The method of  claim 14 , further comprising:
 identifying a sensitivity score, the sensitivity score indicating a level of conservatism in determining the aggregate rating; and   determining the aggregate rating for each document based on the rating information associated with each document, the trust scores associated with each document, and the sensitivity score.   
   
   
       17 . The method of  claim 16 , further comprising:
 identifying a preferred aggregate rating to be associated with the request; and   
   
   
       18 . The method of  claim 17 , wherein ranking the documents based on the evaluation information in response to the request comprises:
 ranking the documents based on the aggregate rating for each document and the preferred aggregate rating.   
   
   
       19 . A computer storage medium encoded with a computer program, the program comprising instructions that when executed by data processing apparatus cause the data processing apparatus to perform operations comprising:
 receiving, at a server, rating information associated with contents of a document from one or more evaluators;   identifying, at the server, a trust score for each of the one or more evaluators, each trust score being a measure of an extent to which a respective evaluator's rating information is considered in determining an aggregate rating, each trust score based on factors associated with the respective evaluator;   identifying a sensitivity score, the sensitivity score indicating a level of conservatism in determining the aggregate rating;   determining the aggregate rating for the document based on the rating information, the trust scores, and the sensitivity score;   receiving a request from a recipient for one or more documents;   identifying a preferred aggregate rating to be associated with the request; and   determining whether to deliver the document in response to the request based on whether the aggregate rating satisfies the preferred aggregate rating.   
   
   
       20 . A computer storage medium encoded with a computer program, the program comprising instructions that when executed by data processing apparatus cause the data processing apparatus to perform operations comprising:
 associating evaluation information with documents, the evaluation information comprising trust information and rating information;   receiving a request at a server for one or more of the documents;   ranking the documents based on the evaluation information in response to the request;   providing the ranked documents; and   updating the trust information based on the ranking.