Using Machine Learning to Predict Outcomes for Documents

Abstract

Evaluations of a document are generated that indicate likelihoods of the document achieving its objectives. The evaluations are based on predictive characteristics of one or more outcomes of the client document that are indicative of whether the document will achieve its objectives. Specifically, a server receives the document from a client device. The server extracts a set of features from the document. The evaluations of the document are generated based on the predictive characteristics for the one or more outcomes of the document. The generated evaluations are provided to the client device such that the document can be optimized to achieve its desired objectives. The optimized document may also be sent to a posting server for posting to a computer network. The known outcomes of the optimized document are collected through reader responses to the document and analyzed to improve evaluations for other documents.

1 . A method of evaluating an electronic document with respect to an objective, comprising:
 receiving the electronic document from a client device via a computer network, the electronic document having content directed toward achieving the obj ective;   extracting a set of features from the content of the electronic document, the extracted features including phrase-related features indicating presence of distinctive phrases in the content of the electronic document, categories corresponding to the distinctive phrases, and the distinctive phrases' levels of association with the corresponding categories;   evaluating the features in the set using one or more machine-learned models that indicate directions and degrees of correlation between the features extracted from the content of the electronic document and the objective to which the content of the document is directed to predict an outcome of the electronic document with respect to the objective, evaluating the features comprising:   assigning weights to the features in the set, a weight assigned to a feature indicating a direction and degree of correlation between the feature and the predicted outcome of the electronic document, and combining the weights of the set of features to predict the outcome of the electronic document; and   providing the predicted outcome for display on a user interface of the client device, wherein providing the predicted outcome comprises identifying the plurality of distinctive phrases in the content of the electronic document and, for each identified distinctive phrase, indicating an influence of the distinctive phrase on the predicted outcome of the electronic document with respect to the objective.   
     
     
         2 . The method of  claim 1 , wherein a sign of a weight assigned to a feature indicates the direction of correlation between the feature and the predicted outcome of the electronic document, and an absolute value of the weight assigned to the feature indicates the degree of correlation between the feature and the predicted outcome. 
     
     
         3 . The method of  claim 1 , wherein a level of association of a distinctive phrase with a corresponding category is represented by a numerical value. 
     
     
         4 . The method of  claim 1 , wherein providing the predicted outcome further comprises identifying a category association of each identified distinctive phrase. 
     
     
         5 . The method of  claim 1 , further comprising:
 establishing a training corpus of electronic documents and associated known outcome data describing known outcomes resulting from postings of the electronic documents on the computer network;   extracting a set of training features from contents of the electronic documents in the training corpus; and   generating the one or more machine-learned models by correlating the sets of training features extracted from the contents of the electronic documents in the training corpus and the associated known outcome data.   
     
     
         6 . The method of  claim 5 , wherein assigning weights to the features in the set is based on coefficients identified through the one or more machine-learned models. The method of  claim 1 , further comprising:
 posting the electronic document on the computer network;   receiving outcome data describing responses to the posting of the electronic document on the computer network with respect to the objective; and   selectively revising the machine-learned models based on the outcome data.   
     
     
         8 . The method of  claim 1 , wherein extracting the set of features from the content of the electronic document further comprises:
 extracting syntactic factors describing a structure of sentences in the content of the electronic document;   extracting structural factors relating to structure and layout of the content of the electronic document; and   extracting semantic factors relating to meaning of the content in the electronic document.   
     
     
         9 . The method of  claim 1 , wherein the electronic document is a recruiting document, the objective relates to demographic information of people responding to the recruiting document, the predicted outcome predicting characteristics of reader responses to the electronic document, and indicating a likelihood that the electronic document will achieve the objective. 
     
     
         10 . The method of  claim 1 , wherein providing the predicted outcome for display on a user interface of the client device further comprises providing a favorability score indicating a likelihood that the electronic document will achieve the objective. 
     
     
         11 . A non-transitory computer-readable storage medium storing computer program instructions executable to perform operations for evaluating an electronic document with respect to an objective, the operations comprising:
 receiving the electronic document from a client device via a computer network, the electronic document having content directed toward achieving the obj ective;   extracting a set of features from the content of the electronic document, the extracted features including phrase-related features indicating presence of distinctive phrases in the content of the electronic document, categories corresponding to the distinctive phrases, and the distinctive phrases' levels of association with the corresponding categories;   evaluating the features in the set using one or more machine-learned models that indicate directions and degrees of correlation between the features extracted from the content of the electronic document and the objective to which the content of the document is directed to predict an outcome of the electronic document with respect to the objective, evaluating the features comprising:   assigning weights to the features in the set, a weight assigned to a feature indicating a direction and degree of correlation between the feature and the predicted outcome of the electronic document, and combining the weights of the set of features to predict the outcome of the electronic document; and   providing the predicted outcome for display on a user interface of the client device, wherein providing the predicted outcome comprises identifying the plurality of distinctive phrases in the content of the electronic document and, for each identified distinctive phrase, indicating an influence of the distinctive phrase on the predicted outcome of the electronic document with respect to the objective.   
     
     
         12 . The computer-readable medium of  claim 11 , wherein a sign of a weight assigned to a feature indicates the direction of correlation between the feature and the predicted outcome of the electronic document, and an absolute value of the weight assigned to the feature indicates the degree of correlation between the feature and the predicted outcome. 
     
     
         13 . The computer-readable medium of  claim 11 , wherein a level of association of a distinctive phrase with a corresponding category is represented by a numerical value. 
     
     
         14 . The computer-readable medium of  claim 11 , wherein providing the predicted outcome further comprises identifying a category association of each identified distinctive phrase. 
     
     
         15 . The computer-readable medium of  claim 11 , wherein the operations further comprise:
 establishing a training corpus of electronic documents and associated known outcome data describing known outcomes resulting from postings of the electronic documents on the computer network;   extracting a set of training features from contents of the electronic documents in the training corpus; and   generating the one or more machine-learned models by correlating the sets of training features extracted from the contents of the electronic documents in the training corpus and the associated known outcome data.   
     
     
         16 . The computer-readable medium of  claim 15 , wherein assigning weights to the features in the set is based on coefficients identified through the one or more machine-learned models. 
     
     
         17 . The computer-readable medium of  claim 11 , wherein the operations further comprise:
 posting the electronic document on the computer network;   receiving outcome data describing responses to the posting of the electronic document on the computer network with respect to the objective; and   selectively revising the machine-learned models based on the outcome data.   
     
     
         18 . The computer-readable medium of  claim 11 , wherein extracting the set of features from the content of the electronic document further comprises:
 extracting syntactic factors describing a structure of sentences in the content of the electronic document;   extracting structural factors relating to structure and layout of the content of the electronic document; and   extracting semantic factors relating to meaning of the content in the electronic document.   
     
     
         19 . The computer-readable medium of  claim 11 , wherein the electronic document is a recruiting document, the objective relates to demographic information of people responding to the recruiting document, the predicted outcome predicting characteristics of reader responses to the electronic document, and indicating a likelihood that the electronic document will achieve the objective. 
     
     
         20 . The computer-readable medium of  claim 11 , wherein providing the predicted outcome for display on a user interface of the client device further comprises providing a favorability score indicating a likelihood that the electronic document will achieve the objective.