Training a machine learning model to determine a predicted time distribution related to electronic communications

Abstract

Techniques for training a machine learning model to determine a predicted time distribution related to electronic communications are discussed herein. The machine learning model is trained based at least in part on time to open data indicative of respective time to open terms that begin at respective transmission times for electronic communications and end at respective electronic communication access event times. Additionally, based at least in part on the predicted time distribution determined by the machine learning model, respective access scores for an electronic communication being accessed via the user device at the respective times are determined to provide a new electronic communication for rendering via an electronic interface of a user device.

1 - 23 . (canceled) 
     
     
         24 . An apparatus, comprising one or more processors and one or more storage devices storing instructions that are operable, when executed by the one or more processors, to cause the one or more processors to:
 transmit respective electronic communications to a plurality of user devices at respective transmission times;   track clickstream data generated by a plurality of user device interactions associated with the respective electronic communications to determine respective electronic communication access event times associated with respective instances for accessing content of the respective electronic communications;   determine time to open data based at least in part on the respective transmission times and the respective electronic communication access event times, wherein the time to open data is indicative of respective time to open terms that begin at the respective transmission times and end at the respective electronic communication access event times;   train, based at least in part on the time to open data, a machine learning model to determine a predicted time distribution of predicted time to open data indicative of respective predicted times associated with an electronic communication being accessed via a user device;   determine, based at least in part on the predicted time distribution, respective access scores for the electronic communication being accessed via the user device at the respective times;   determine, based at least in part on the respective access scores associated with the predicted time distribution, a relevance score indicative of a degree of relevance for a new electronic communication comprising communication content with respect to a user identifier associated with the user device; and   in response to a determination that the relevance score satisfies a defined relevance score threshold, transmit the new electronic communication to the user device to render the communication content via an electronic interface of the user device.   
     
     
         25 . The apparatus of  claim 24 , wherein the one or more storage devices store instructions that are operable, when executed by the one or more processors, to further cause the one or more processors to:
 determine, based at least in part on the respective access scores associated with the predicted time distribution, a freshness score that defines access probability for the new electronic communication comprising the communication content; and   apply the freshness score to a prior version of the relevance score to determine the relevance score.   
     
     
         26 . The apparatus of  claim 24 , wherein the one or more storage devices store instructions that are operable, when executed by the one or more processors, to further cause the one or more processors to:
 determine, based at least in part on the respective access scores associated with the predicted time distribution, a freshness score that defines access probability for the new electronic communication comprising the communication content;   apply the freshness score associated with the predicted time distribution to the relevance score to generate an updated relevance score;   rank a plurality of communication content sets comprising at least the communication content based on respective updated relevance scores to generate ranked communication content; and   configure the new electronic communication with a subset of the ranked communication content.   
     
     
         27 . The apparatus of  claim 24 , wherein the one or more storage devices store instructions that are operable, when executed by the one or more processors, to further cause the one or more processors to:
 determine the relevance score in response to a search request provided by the user device.   
     
     
         28 . The apparatus of  claim 24 , wherein the one or more storage devices store instructions that are operable, when executed by the one or more processors, to further cause the one or more processors to:
 transform the clickstream data into one or more feature vectors; and   train the machine learning model based at least in part on the one or more feature vectors.   
     
     
         29 . The apparatus of  claim 24 , wherein the one or more storage devices store instructions that are operable, when executed by the one or more processors, to further cause the one or more processors to:
 train the machine learning model to determine a parametric access probability function associated with the predicted time distribution.   
     
     
         30 . The apparatus of  claim 24 , wherein the one or more storage devices store instructions that are operable, when executed by the one or more processors, to further cause the one or more processors to:
 train the machine learning model to determine a cumulative distribution function associated with the predicted time distribution.   
     
     
         31 . The apparatus of  claim 24 , wherein the one or more storage devices store instructions that are operable, when executed by the one or more processors, to further cause the one or more processors to:
 generate scaling factor data for relevance scores related to electronic communications based at least in part on the respective access scores associated with the predicted time distribution; and   store the scaling factor data in one or more data tables for employment by an online networked processor that processes real-time search requests provided by user devices.   
     
     
         32 . A computer-implemented method, comprising:
 transmitting, by a computing device comprising a processor, respective electronic communications to a plurality of user devices at respective transmission times;   tracking, by the computing device, clickstream data generated by a plurality of user device interactions associated with the respective electronic communications to determine respective electronic communication access event times associated with respective instances for accessing content of the respective electronic communications;   determining, by the computing device, time to open data based at least in part on the respective transmission times and the respective electronic communication access event times,   wherein the time to open data is indicative of respective time to open terms that begin at the respective transmission times and end at the respective electronic communication access event times;   training, by the computing device and based at least in part on the time to open data, a machine learning model to determine a predicted time distribution of predicted time to open data indicative of respective predicted times associated with an electronic communication being accessed via a user device;   determining, by the computing device and based at least in part on the predicted time distribution, respective access scores for the electronic communication being accessed via the user device at the respective times;   determining, by the computing device and based at least in part on the respective access scores associated with the predicted time distribution, a relevance score indicative of a degree of relevance for a new electronic communication comprising communication content with respect to a user identifier associated with the user device; and   in response to a determination that the relevance score satisfies a defined relevance score threshold, transmitting, by the computing device, the new electronic communication to the user device to render the communication content via an electronic interface of the user device.   
     
     
         33 . The computer-implemented method of  claim 32 , further comprising:
 determining, by the computing device and based at least in part on the respective access scores associated with the predicted time distribution, a freshness score that defines access probability for the new electronic communication comprising the communication content; and   applying, by the computing device, the freshness score to a prior version of the relevance score to determine the relevance score.   
     
     
         34 . The computer-implemented method of  claim 32 , further comprising:
 determining, by the computing device and based at least in part on the respective access scores associated with the predicted time distribution, a freshness score that defines access probability for the new electronic communication comprising the communication content;   applying, by the computing device, the freshness score associated with the predicted time distribution to the relevance score to generate an updated relevance score;   ranking, by the computing device, a plurality of communication content sets comprising at least the communication content based on respective updated relevance scores to generate ranked communication content; and   configuring, by the computing device, the new electronic communication with a subset of the ranked communication content.   
     
     
         35 . The computer-implemented method of  claim 32 , wherein the determining the relevance score comprises determining the relevance score in response to a search request provided by the user device. 
     
     
         36 . The computer-implemented method of  claim 32 , further comprising:
 transforming, by the computing device, the clickstream data into one or more feature vectors; and   training, by the computing device, the machine learning model based at least in part on the one or more feature vectors.   
     
     
         37 . The computer-implemented method of  claim 32 , further comprising:
 training, by the computing device, the machine learning model to determine a parametric access probability function associated with the predicted time distribution.   
     
     
         38 . The computer-implemented method of  claim 32 , further comprising:
 training, by the computing device, the machine learning model to determine a cumulative distribution function associated with the predicted time distribution.   
     
     
         39 . The computer-implemented method of  claim 32 , further comprising:
 generating, by the computing device, scaling factor data for relevance scores related to electronic communications based at least in part on the respective access scores associated with the predicted time distribution; and   storing, by the computing device, the scaling factor data in one or more data tables for employment by an online networked processor that processes real-time search requests provided by user devices.   
     
     
         40 . A computer program product, stored on a computer readable medium, comprising instructions that when executed by one or more computers cause the one or more computers to:
 transmit respective electronic communications to a plurality of user devices at respective transmission times;   track clickstream data generated by a plurality of user device interactions associated with the respective electronic communications to determine respective electronic communication access event times associated with respective instances for accessing content of the respective electronic communications;   determine time to open data based at least in part on the respective transmission times and the respective electronic communication access event times, wherein the time to open data is indicative of respective time to open terms that begin at the respective transmission times and end at the respective electronic communication access event times;   train, based at least in part on the time to open data, a machine learning model to determine a predicted time distribution of predicted time to open data indicative of respective predicted times associated with an electronic communication being accessed via a user device;   determine, based at least in part on the predicted time distribution, respective access scores for the electronic communication being accessed via the user device at the respective times;   determine, based at least in part on the respective access scores associated with the predicted time distribution, a relevance score indicative of a degree of relevance for a new electronic communication comprising communication content with respect to a user identifier associated with the user device; and   in response to a determination that the relevance score satisfies a defined relevance score threshold, transmit the new electronic communication to the user device to render the communication content via an electronic interface of the user device.   
     
     
         41 . The computer program product of  claim 40 , further comprising instructions that when executed by the one or more computers cause the one or more computers to:
 determine, based at least in part on the respective access scores associated with the predicted time distribution, a freshness score that defines access probability for the new electronic communication comprising the communication content; and   apply the freshness score to a prior version of the relevance score to determine the relevance score.   
     
     
         42 . The computer program product of  claim 40 , further comprising instructions that when executed by the one or more computers cause the one or more computers to:
 determine, based at least in part on the respective access scores associated with the predicted time distribution, a freshness score that defines access probability for the new electronic communication comprising the communication content;   apply the freshness score associated with the predicted time distribution to the relevance score to generate an updated relevance score;   rank a plurality of communication content sets comprising at least the communication content based on respective updated relevance scores to generate ranked communication content; and   configure the new electronic communication with a subset of the ranked communication content.   
     
     
         43 . The computer program product of  claim 40 , further comprising instructions that when executed by the one or more computers cause the one or more computers to:
 determine the relevance score in response to a search request provided by the user device.