Web content organization and presentation techniques

Abstract

An online system that identifies allocations of both organic and promoted content on a given page. The allocations of page content are compared against one another and configured to prioritize for overall utility based on objective factors that quantify a page “look and feel” as measured by machine learning models. The page allocations are operated on an automatic and continuous basis for each user viewing the page. In some embodiments, the page content allocations are based on individual viewing users stored characteristics.

1 . A method to place a promoted content in a webpage comprising:
 sending a request for a content to an organic system and a promoted system,
 wherein the promoted system provides a bid for placement of the promoted content associated with the promoted system, and 
 wherein the organic system provides an organic content without placing a bid for placement of the organic content; 
   receiving multiple organic contents and multiple promoted contents from the organic system and the promoted system, respectively;   computing a quality score for each organic content among the multiple organic contents and for each promoted content among the multiple promoted contents,
 wherein the quality score represents a probability that a user engages with the each organic content and the each promoted content; 
   creating a first arrangement by allocating multiple content slots associated with the webpage to the multiple organic contents and the multiple promoted contents based on the quality score associated with the each organic content;   determining a utility value associated with a content slot among the multiple content slots by determining a user experience cost between showing the organic content in the content slot and showing the promoted content in the content slot;   obtaining a termination condition indicating whether a minimum spacing between the multiple content slots has been achieved;   until the termination condition is satisfied, iteratively performing:
 creating an arrangement associated with the webpage,
 wherein the arrangement has different promoted content compared to a previous arrangement, 
 wherein the different promoted content includes different spacing between promoted content contained in the arrangement and promoted content in the previous arrangement; 
 
 calculating a webpage utility score associated with the arrangement based on the user experience cost; 
   comparing the multiple webpage utility scores to obtain the highest score; and   selecting a webpage having the highest score to display to the user.   
     
     
         2 . A method comprising:
 sending a request for a content to an organic system and a promoted system,
 wherein the promoted system provides a bid for placement of a promoted content associated with the promoted system, and 
 wherein the organic system provides an organic content without placing a bid for placement of the organic content; 
   receiving multiple organic contents and multiple promoted contents from the organic system and the promoted system, respectively;   computing a quality score for each organic content among the multiple organic contents and for each promoted content among the multiple promoted contents,
 wherein the quality score represents a probability that a user engages with the each organic content and the each promoted content; 
   creating a first arrangement by allocating multiple content slots associated with a webpage to the multiple organic contents based on the quality score associated with the each organic content; and   reordering the multiple content slots by moving the promoted content among the multiple promoted contents based on the quality score associated with the promoted content by:
 determining an auction bid indicating an increase in a probability that the user engages with the promoted content when the promoted content is reordered to a different content slot; and 
 based on the auction bid, moving the promoted content. 
   
     
     
         3 . The method of  claim 2 , comprising:
 obtaining a user experience cost and an impact control,
 wherein the user experience cost represents a difference in a user's experience between viewing the organic content in a content slot on the webpage and viewing the promoted content in the content slot on the webpage, 
 wherein the impact control represents an effect the difference in the user's experience has on a webpage utility score; 
   computing a first webpage utility score of the first arrangement based on the auction bid, the user experience cost and the impact control;   creating a second arrangement by reallocating the multiple content slots to increase a promoted content load;   computing a second webpage utility score of the second arrangement;   comparing the first webpage utility score and the second webpage utility score; and   selecting a webpage having a higher webpage utility score to display to the user.   
     
     
         4 . The method of  claim 2 , comprising:
 determining a utility value associated with a content slot among the multiple content slots by determining a user experience cost between showing the organic content in the content slot and showing the promoted content in the content slot;   obtaining a termination condition indicating whether minimum spacing between the multiple content slots has been achieved;   until the termination condition is satisfied, iteratively performing:
 creating an arrangement associated with the webpage,
 wherein the arrangement has different promoted content compared to previous arrangement, 
 wherein the different promoted content includes different spacing between promoted content contained in the arrangement and promoted content in the previous arrangement; 
 
 calculating a webpage utility score associated with the arrangement based on the user experience cost; 
   comparing the multiple webpage utility scores to obtain the highest score; and   selecting a webpage having the highest score to display to the user.   
     
     
         5 . The method of  claim 2 , wherein computing the quality score comprises:
 gathering data associated with the user comprising an interaction with a previously promoted content,
 wherein the interaction includes clicking the previously promoted content, sharing the previously promoted content, indicating a preference for the previously promoted content, or posting a comment regarding the previously promoted content; 
   training a machine learning model based on the gathered data; and   computing the quality score using the trained machine learning model.   
     
     
         6 . The method of  claim 2 , wherein computing the quality score comprises:
 gathering data associated with the user comprising an interaction with a previously promoted content,
 wherein the interaction includes clicking the previously promoted content, sharing the previously promoted content, indicating a preference for the previously promoted content, or posting a comment regarding the previously promoted content; 
   generating training data based on the data associated with the user by standardizing the data associated with the user;   based on the training data, predicting a likelihood that a type of benefit to the promoted system occurs for a type of action performed by the user in response to being presented with the promoted content.   
     
     
         7 . The method of  claim 2 , wherein computing the quality score comprises:
 gathering data associated with the user comprising an interaction with a previously promoted content;   generating training data based on the data associated with the user by standardizing the data associated with the user;   based on the training data, predicting a likelihood that a type of benefit to the promoted system occurs for a type of action performed by the user within a timeframe in response to the user being presented with the promoted content by:
 determining a total number of impressions of the promoted content presented to multiple users; 
 obtaining a first probability by determining how many of the total number of impressions result in the type of action being performed by the multiple users; 
 obtaining a second probability by determining a number of the type of action performed by the multiple users and a number of the type of benefit occurring within the timeframe; and 
 determining the likelihood based on the first probability and the second probability. 
   
     
     
         8 . The method of  claim 2 , wherein computing the quality score comprises:
 obtaining a dictionary comprising multiple topics associated with the multiple promoted contents;   classifying the promoted content into a first subset of topics among the multiple topics;   generating a first subset of probabilities indicating a correspondence between the promoted content and a topic in the first subset of topics;   obtaining a second subset of topics among the multiple topics and a second subset of probabilities,
 wherein each probability in the second subset of probabilities indicates a user's affinity toward a corresponding topic among the second subset of topics; 
   computing the quality score based on a first measure of similarity between the first subset of probabilities and the second subset of probabilities or a second measure of similarity between the first subset of topics and the second subset of topics.   
     
     
         9 . The method of  claim 8 , wherein computing the quality score based on the first measure or the second measure of similarity comprises:
 calculating a dot product between the first subset of probabilities and the second subset of probabilities.   
     
     
         10 . The method of  claim 8 , wherein computing the quality score based on the first measure or the second measure of similarity comprises:
 calculating a number of overlapping topics between the first subset of topics and the second subset of topics.   
     
     
         11 . The method of  claim 2 , wherein computing the quality score comprises:
 determining that the promoted content contains an image, a sequence of images, or an audio;   determining a degradation of the promoted content; and   calculating the quality score based on the degradation of the promoted content, wherein the lower the degradation, the higher the quality score.   
     
     
         12 . The method of  claim 2 , wherein computing the quality score comprises:
 determining the promoted system providing the promoted content; and   calculating the quality score based on the promoted system.   
     
     
         13 . The method of  claim 2 , wherein computing the quality score comprises:
 determining a quality of a destination to which the promoted content may connect; and   computing the quality score based on the quality of the destination.   
     
     
         14 . The method of  claim 2 , wherein computing the quality score comprises:
 obtaining a dictionary comprising multiple topics associated with the multiple promoted contents;   classifying the promoted content into a first subset of topics among the multiple topics;   generating a first subset of probabilities indicating a correspondence between the promoted content and a topic in the first subset of topics;   classifying the request into a second subset of topics among the multiple topics;   generating a second subset of probabilities indicating a correspondence between the request and a topic in the second subset of topics; and   computing the quality score based on a first measure of similarity between the first subset of probabilities and the second subset of probabilities or a second measure of similarity between the first subset of topics and the second subset of topics.   
     
     
         15 . The method of  claim 14 , wherein computing the quality score based on the first measure or the second measure of similarity comprises:
 calculating a dot product between the first subset of probabilities and the second subset of probabilities.   
     
     
         16 . The method of  claim 14 , wherein computing the quality score based on the first measure or the second measure of similarity comprises:
 calculating a number of overlapping topics between the first subset of topics and the second subset of topics.   
     
     
         17 . A system comprising:
 at least one hardware processor; and   at least one non-transitory memory storing instruction, which, when executed by the at least one hardware processor, causes the system to:
 send a request for a content to an organic system and a promoted system,
 wherein the promoted system provides a bid for placement of a promoted content associated with the promoted system, and 
 wherein the organic system provides an organic content without placing a bid for placement of the organic content; 
 
 receive multiple organic contents and multiple promoted contents from the organic system and the promoted system, respectively; 
 compute a quality score for each organic content among the multiple organic contents and for each promoted content among the multiple promoted contents,
 wherein the quality score represents a probability that a user engages with the each organic content and the each promoted content; 
 
 create a first arrangement by allocating multiple content slots associated with a webpage to the multiple organic contents based on the quality score associated with the each organic content; and 
 reorder the multiple content slots by moving the promoted content among the multiple promoted contents based on the quality score associated with the promoted content by:
 determining an auction bid indicating an increase in a probability that the user engages with the promoted content when the promoted content is reordered to a different content slot; and 
 based on the auction bid, moving the promoted content. 
 
   
     
     
         18 . The system of  claim 17 , comprising the instructions to:
 obtain a user experience cost and an impact control,
 wherein the user experience cost represents a difference in a user's experience between viewing the organic content in a content slot on the webpage and viewing the promoted content in the content slot on the webpage, 
 wherein the impact control represents an effect the difference in the user's experience has on a webpage utility score; 
   compute a first webpage utility score of the first arrangement based on the auction bid, the user experience cost and the impact control;   create a second arrangement by reallocating the multiple content slots to increase a promoted content load;   compute a second webpage utility score of the second arrangement;   compare the first webpage utility score and the second webpage utility score; and   select a webpage having a higher webpage utility score to display to the user.   
     
     
         19 . The system of  claim 17 , comprising the instructions to:
 determine a utility value associated with a content slot among the multiple content slots by determining a user experience cost between showing the organic content in the content slot and showing the promoted content in the content slot;   obtain a termination condition indicating whether minimum spacing between the multiple content slots has been achieved;   until the termination condition is satisfied, iteratively perform:
 creating an arrangement associated with the webpage,
 wherein the arrangement has different promoted content compared to the previous arrangement; 
 
 calculating a webpage utility score associated with the arrangement; 
   compare the multiple webpage utility scores to obtain the highest score; and   select a webpage having the highest score to display to the user.   
     
     
         20 . The system of  claim 17 , the instructions to compute the quality score comprising the instructions to:
 gather data associated with the user comprising an interaction with a previously promoted content,
 wherein the interaction includes clicking the previously promoted content, sharing the previously promoted content, indicating a preference for the previously promoted content, or posting a comment regarding the previously promoted content; 
   train a machine learning model based on the gathered data; and   compute the quality score using the trained machine learning model.   
     
     
         21 . The system of  claim 17 , the instructions to compute the quality score comprising instructions to:
 gather data associated with the user comprising an interaction with a previously promoted content,
 wherein the interaction includes clicking the previously promoted content, sharing the previously promoted content, indicating a preference for the previously promoted content, or posting a comment regarding the previously promoted content; 
   generate training data based on the data associated with the user by standardizing the data associated with the user;   based on the training data, predict a likelihood that a type of benefit to the promoted system occurs for a type of action performed by the user in response to being presented with the promoted content.   
     
     
         22 . The system of  claim 17 , the instructions to compute the quality score comprising instructions to:
 gather data associated with the user comprising an interaction with a previously promoted content;   generate training data based on the data associated with the user by standardizing the data associated with the user;   based on the training data, predict a likelihood that a type of benefit to the promoted system occurs for a type of action performed by the user within a timeframe in response to the user being presented with the promoted content by:
 determining a total number of impressions of the promoted content presented to multiple users; 
 obtaining a first probability by determining how many of the total number of impressions result in the type of action being performed by the multiple users; 
 obtaining a second probability by determining a number of the type of action performed by the multiple users and a number of the type of benefit occurring within the timeframe; and 
 determining the likelihood based on the first probability and the second probability. 
   
     
     
         23 . The system of  claim 17 , the instructions to compute the quality score comprising the instructions to:
 obtain a dictionary comprising multiple topics associated with the multiple promoted contents;   classify the promoted content into a first subset of topics among the multiple topics;   generate a first subset of probabilities indicating a correspondence between the promoted content and a topic in the first subset of topics;   obtain a second subset of topics among the multiple topics and a second subset of probabilities,
 wherein each probability in the second subset of probabilities indicates a user's affinity toward a corresponding topic among the second subset of topics; 
   compute the quality score based on a first measure of similarity between the first subset of probabilities and the second subset of probabilities or a second measure of similarity between the first subset of topics and the second subset of topics.   
     
     
         24 . The system of  claim 23 , the instructions to compute the quality score based on the first measure or the second measure of similarity comprising the instructions to:
 calculate a dot product between the first subset of probabilities and the second subset of probabilities.   
     
     
         25 . The system of  claim 23 , the instructions to compute the quality score based on the first measure or the second measure of similarity comprising the instructions to:
 calculate a number of overlapping topics between the first subset of topics and the second subset of topics.   
     
     
         26 . The system of  claim 17 , the instructions to compute the quality score comprising the instructions to:
 determine that the promoted content contains an image, a sequence of images, or an audio;   determine a degradation of the promoted content; and   calculate the quality score based on the degradation of the promoted content, wherein the lower the degradation, the higher the quality score.   
     
     
         27 . The system of  claim 17 , the instructions to compute the quality score comprising the instructions to:
 determine the promoted system providing the promoted content; and   calculate the quality score based on the promoted system.   
     
     
         28 . The system of  claim 17 , the instructions to compute the quality score comprising the instructions to:
 determine a quality of a destination to which the promoted content may connect; and   compute the quality score based on the quality of the destination.   
     
     
         29 . The system of  claim 17 , the instructions to compute the quality score comprising the instructions to:
 obtain a dictionary comprising multiple topics associated with the multiple promoted contents;   classify the promoted content into a first subset of topics among the multiple topics;   generate a first subset of probabilities indicating a correspondence between the promoted content and a topic in the first subset of topics;   classify the request into a second subset of topics among the multiple topics;   generate a second subset of probabilities indicating a correspondence between the request and a topic in the second subset of topics; and   compute the quality score based on a first measure of similarity between the first subset of probabilities and the second subset of probabilities or a second measure of similarity between the first subset of topics and the second subset of topics.