Quality scoring system for advertisements and content in an online system

Abstract

A server system of an online information system displays advertising items and content items retrieved from storage devices as a stream viewable by a user on a user device. The advertisement items and the content items are ordered in the stream by a ranking score computed for each of the advertisement items and each of the content items. A quality scoring system determines an affinity score between a user and a present content item based on features of the present content item matching user profile parameters associated with the user and identifies post-interaction satisfaction with a prior content item. The quality scoring system determines a quality score based on the affinity score and the post-interaction satisfaction. The quality score is used for ordering items in the stream. The server system transmits a web page including the stream to a user device over a network. In this manner, advertising items and content items compete in a unified marketplace for inclusion in the stream for viewing by the end user.

1 . A method for determining a quality score for a content item in a streaming media feed that includes content items and advertisements, comprising:
 identifying, by a positioning system, a present geographic location of a mobile device;   determining, by a processor communicatively coupled to the positioning system, an affinity score between the mobile device and a present content item in a streaming media feed, based on text in the present content item and the present geographic location of the mobile device;   identifying post-interaction satisfaction with a prior content item that matches categorically the present content item; and   determining, by the processor, a quality score based on the affinity score and the post-interaction satisfaction; and   displaying the content items with the advertisements in the streaming media feed based on the determined quality score, wherein a higher quality score corresponds with a higher position in the streaming media feed.   
     
     
         2 . The method of  claim 1 , wherein the determining of the affinity score further includes:
 determining a first probability, which is a probability a user of the mobile device will select to view the present content item, based on features of the present content item matching user profile parameters associated with the user;   determining a second probability, which is a probability the user will select to view a content item, in general, in a streaming media feed; and   determining the affinity score based on a relationship between the first determined probability and the second determined probability.   
     
     
         3 . The method of  claim 2 , wherein the determining the first or the second probability uses a machine learning technique. 
     
     
         4 . The method of  claim 2 , wherein the determining of the first or the second probability uses a boosted decision tree. 
     
     
         5 . The method of  claim 2 , wherein the determining the quality score based on the affinity score and the post-interaction satisfaction further comprises:
 deriving a post-interaction satisfaction score based on the identified post-interaction satisfaction; and   determining the quality score by computing a product of the affinity score and the post-interaction satisfaction score.   
     
     
         6 . The method of  claim 1 , wherein the affinity score is independent of a position of the present content item in a streaming media feed. 
     
     
         7 . The method of  claim 1 , wherein the affinity score is independent of a configuration of the present content item in a streaming media feed. 
     
     
         8 . The method of  claim 2 , wherein the determining the quality score is further based on a popularity score. 
     
     
         9 . The method of  claim 8 , wherein the popularity score is based on a click through rate for the present content item within a window of time. 
     
     
         10 . The method of  claim 1 , wherein the present content item is a first content item, wherein a second content item is an article, wherein the first and the second content items include similar subject matter categorically, and wherein the instructions to determine the affinity score include instructions to determine an affinity score of the second content item based on the affinity score of the first content item. 
     
     
         11 . The method of  claim 10 , wherein the similarity of the subject matter categorically is identified by matching metadata elements embedded in the first and the second content items, and wherein the matching metadata elements are categories of a content taxonomy. 
     
     
         12 . The method of  claim 10 , wherein the similarity of the subject matter categorically is identified by matching metadata elements embedded in the first and the second content items, and wherein the matching metadata elements are associations with a webpage of a series of webpages provided by a web content provider. 
     
     
         13 . The method of  claim 12 , wherein the webpage is an online encyclopedia or dictionary entry. 
     
     
         14 . A system for unified display of a plurality of revenue and non-revenue generating items, the system comprising:
 a first database comprising a plurality of revenue generating items and historical bid values generated for the plurality of revenue generating items;   a second database comprising a plurality of non-revenue generating items; and   a ranking engine with at least one processor configured to calculate a ranking score for each of the plurality of revenue generating items using the historical bid values of the first database, and configured to calculate a ranking score for each of the plurality of non-revenue generating items, wherein the ranking score for the non-revenue generating items correlates with the ranking score for the revenue-generating items such that the display of the revenue generating items with the non-revenue generating items is ordered based on respective ranking scores.   
     
     
         15 . The system of  claim 14 , further comprising:
 displaying the revenue generating items in a streaming media feed based on the determined ranking score, wherein a higher ranking score corresponds with a higher position in the streaming media feed.   
     
     
         16 . The system of  claim 15 , wherein the instructions to determine the ranking score are further based on an affinity score and a post-interaction satisfaction and further comprise:
 deriving a post-interaction satisfaction score based on an identified post-interaction satisfaction; and   determining the ranking score by computing a product of the affinity score and the post-interaction satisfaction score.   
     
     
         17 . The system of  claim 16 , wherein the affinity score is independent of a position in the streaming media feed. 
     
     
         18 . The system of  claim 16 , wherein the affinity score determination further comprises:
 determining a first probability, which is a probability the user will select to interact with a content item in a streaming media feed;   determining a second probability, which is a probability the user will select to interact with a content item, in general, in the streaming media feed, wherein the relationship between the first determined probability and the second determined probability is a ratio; and   determining the affinity score based on a relationship between the first determined probability and the second determined probability.   
     
     
         19 . A system for determining a quality score for an advertisement a streaming media feed, comprising:
 a processor; and   memory, the memory communicatively coupled to the processor, the memory comprising instructions executable by the processor for:
 identifying, by a positioning system, a present geographic location of a mobile device; 
 determining, by a processor communicatively coupled to the positioning system, an affinity score between the mobile device and a present content item in a streaming media feed, based on text in the present content item and the present geographic location of the mobile device; 
 identifying post-interaction satisfaction with a prior content item that matches categorically the present content item; and 
 determining, by the processor, a quality score based on the affinity score and the post-interaction satisfaction; and 
 displaying the content items with the advertisements in the streaming media feed based on the determined quality score, wherein a higher quality score corresponds with a higher position in the streaming media feed. 
   
     
     
         20 . The system of  claim 20 , wherein the determining of the affinity score further includes:
 determining a first probability, which is a probability a user of the mobile device will select to view the present content item, based on features of the present content item matching user profile parameters associated with the user;   determining a second probability, which is a probability the user will select to view a content item, in general, in a streaming media feed; and   determining the affinity score based on a relationship between the first determined probability and the second determined probability.