Client-side generation of lossless object model representations of dynamic webpages

Abstract

The disclosed technology provides for generating a lossless representation of a webpage on a client device. A method can include rendering, by a client device, a webpage to be presented in a web browser at the client device, receiving an indication of user input to run a content check on the rendered webpage, calling, in response to receiving the indication of user input, a scraping function to scrape a document object model (“DOM”) of the rendered webpage in the web browser at the client device, generating a lossless representation of the scraped DOM of the rendered webpage, and transmitting, to a web content analysis system, the lossless representation of the scraped DOM to be used when running the content check for the rendered webpage. The lossless representation of the scraped DOM can be a flat object model (“FlatOM”) of the DOM of the rendered webpage.

What is claimed is: 
     
       1. A method for generating a lossless representation of a webpage on a client device, the method comprising:
 rendering, by a client device, a webpage to be presented in a web browser at the client device; 
 receiving, by the client device, an indication of user input to run a content check on the rendered webpage; 
 calling, by the client device and in response to receiving the indication of user input, a scraping function to scrape a document object model (“DOM”) of the rendered webpage in the web browser at the client device; 
 scraping, by the client device and based on calling the scraping function, a cascading style sheet object model (“CSSOM”) of the rendered webpage in the web browser at the client device; 
 generating, by the client device, a lossless static object model of the scraped DOM and the scraped CSSOM for the rendered webpage, wherein the lossless static object model combines attributes and information of nodes that are traversed by scraping the DOM and scraping the CSSOM; and 
 transmitting, by the client device and to a web content analysis system, the lossless static object model to be used by the web content analysis system to losslessly reproduce, based on the combined attributes and information of the traversed nodes, the rendered webpage in a same rendering as in the web browser at the client device for running the content check on the rendered webpage. 
 
     
     
       2. The method of  claim 1 , wherein the rendered webpage is a preview of the webpage. 
     
     
       3. The method of  claim 1 , wherein the indication of user input includes a reference pointer to a portion of the webpage DOM for which to run the content check. 
     
     
       4. The method of  claim 3 , wherein the reference is at least one of a link to a preview of the rendered webpage and a DOM root node. 
     
     
       5. The method of  claim 3 , further comprising:
 calling, by the client device and based on the reference pointer, the scraping function to scrape the portion of the webpage DOM that corresponds to the reference pointer; and 
 responsively generating, by the client device, a lossless static object model of the scraped portion of the DOM and the scraped CSSOM. 
 
     
     
       6. The method of  claim 1 , further comprising:
 receiving, by the client device and from a computer system, a script including code for scraping a DOM of a webpage at a client device; 
 exposing, by the client device, the code as the scraping function; and 
 responsively calling, by the client device, the scraping function to scrape the DOM of the rendered webpage in the web browser at the client device. 
 
     
     
       7. The method of  claim 1 , wherein the lossless static object model of the scraped DOM and the scraped CSSOM is a flat object model (“FlatOM”) of the DOM and the CSSOM of the rendered webpage. 
     
     
       8. The method of  claim 1 , wherein the lossless static object model of the scraped DOM and the scraped CSSOM is ingested directly into a content check pipeline of the web content analysis system. 
     
     
       9. The method of  claim 8 , wherein ingesting the lossless static object model of the scraped DOM and the scraped CSSOM directly into the content check pipeline comprises bypassing internal rendering of the DOM and the CSSOM at the web content analysis system. 
     
     
       10. The method of  claim 1 , wherein the content check is a pre-publish check of the rendered webpage before the rendered webpage is published and presented in web browsers of user devices. 
     
     
       11. The method of  claim 1 , further comprising:
 receiving, by the client device and from the web content analysis system, results from running the content check for the rendered webpage using the lossless static object model of the scraped DOM and the scraped CSSOM; and 
 outputting, by the client device, the results in a plugin to be presented with the rendered webpage in the web browser at the client device. 
 
     
     
       12. The method of  claim 1 , wherein the webpage is rendered in a content management service (“CMS”) interface presented in the web browser at the client device. 
     
     
       13. The method of  claim 12 , wherein a CMS plugin is presented with the rendered webpage in the CMS interface, the CMS plugin including results from the web content analysis system running the content check on the rendered webpage. 
     
     
       14. The method of  claim 12 , wherein the rendered webpage is an editable version of the webpage. 
     
     
       15. A system for generating a lossless representation of a webpage on a client device, the system comprising:
 a client device configured to render webpage content in a web browser at the client device; and 
 a web content analysis system in communication with the client device and configured to:
 run at least one content check based on analyzing content of the webpage that is rendered at the client device; and 
 generate a plugin for deployment at the client device, wherein the plugin is configured to provide integration between the client device and the web content analysis system, 
 
 wherein the client device is configured to perform operations comprising:
 rendering the webpage to be presented in the web browser at the client device with the plugin; 
 receiving an indication of user input to run a content check on the rendered webpage; 
 calling, in response to receiving the indication of user input, a scraping function to scrape a DOM of the rendered webpage in the web browser at the client device, wherein the scraping function is provided by the web content analysis system via the plugin; 
 scraping, based on calling the scraping function, a cascading style sheet object model (“CSSOM”) of the rendered webpage in the web browser at the client device; 
 generating a lossless static object model of the scraped DOM and the scraped CSSM for the rendered webpage, wherein the lossless static object model combines attributes and information of nodes that are traversed by scraping the DOM and scraping the CSSOM; and 
 transmitting, to the web content analysis system, the lossless static object model to be used by the web content analysis system to losslessly reproduce, based on the combined attributes and information of the traversed nodes, the rendered webpage in a same rendering as in the web browser at the client device for running the at least one content check on the rendered webpage. 
 
 
     
     
       16. The system of  claim 15 , wherein the client device performs operations further comprising:
 receiving, from a computer system, a script including code for scraping a DOM of a webpage at a client device; 
 exposing the code as the scraping function; and 
 responsively calling the scraping function to scrape the DOM of the rendered webpage in the web browser at the client device. 
 
     
     
       17. The system of  claim 15 , wherein the at least one content check is a pre-publish check of the rendered webpage before the rendered webpage is published and presented in web browsers of user devices. 
     
     
       18. The system of  claim 15 , wherein the client device performs operations further comprising:
 receiving, from the web content analysis system, results from running the at least one content check for the rendered webpage using the lossless static object model of the scraped DOM and the scraped CSSOM; and 
 outputting the results in the plugin to be presented with the rendered webpage in the web browser at the client device. 
 
     
     
       19. The system of  claim 15 , wherein:
 the webpage is rendered in a content management service (“CMS”) interface presented in the web browser at the client device, and 
 the plugin is presented with the rendered webpage in the CMS interface, the plugin including results from the web content analysis system running the at least one content check on the rendered webpage. 
 
     
     
       20. The system of  claim 15 , wherein the lossless static object model of the scraped DOM and the scraped CSSOM is ingested directly into a content check pipeline of the web content analysis system, wherein the ingesting comprises bypassing internal rendering of the DOM and the CSSOM at the web content analysis system.