Methods and apparatus to supplement web crawling with cached data from distributed devices

Abstract

Methods and apparatus to supplement web crawling with cached data from distributed devices are disclosed. An example apparatus includes memory, instructions in the apparatus, and processor circuitry to execute the instructions to identify a first webpage, access a set of cached webpages cached in a panelist device, when the first webpage is in the set of cached webpages: identify a first language setting used by the panelist device when the first webpage was accessed, and transmit a first number of requests for the first webpage, respective ones of the requests in the first number identifying different language settings, the requests in the first number of requests skipping the first language setting used by the panelist device when the first webpage was accessed, thereby reducing a total number of requests sent to the first webpage.

What is claimed is: 
     
         1 . An apparatus comprising:
 interface circuitry; and   processor circuitry to execute instructions to:
 identify a first webpage; 
 access a set of cached webpages cached in a panelist device; 
 when the first webpage is in the set of cached webpages:
 identify a first language setting used by the panelist device when the first webpage was accessed; and 
 transmit a first number of requests for the first webpage, respective ones of the requests in the first number identifying different language settings, the requests in the first number of requests skipping the first language setting used by the panelist device when the first webpage was accessed, thereby reducing a total number of requests sent to the first webpage. 
 
   
     
     
         2 . The apparatus of  claim 1 , wherein the set of cached webpages is accessed from a browser history of the panelist device. 
     
     
         3 . The apparatus of  claim 2 , wherein the browser history is native to a browser of the panelist device. 
     
     
         4 . The apparatus of  claim 1 , wherein the set of cached webpages is accessed from a log of webpages in the panelist device. 
     
     
         5 . The apparatus of  claim 1 , wherein the instructions, when executed, cause the processor circuitry to:
 when the first webpage is in the set of cached webpages:
 identify a first browser type used by the panelist device to access the first webpage; and 
 transmit a second number of requests for the first webpage, respective ones of the requests in the second number identifying different browser types, the requests in the second number of requests skipping the first browser type used by the panelist device to access the first webpage, thereby reducing the total number of requests sent to the first webpage. 
   
     
     
         6 . The apparatus of  claim 5 , wherein the instructions, when executed, cause the processor circuitry to:
 when the first webpage is in the set of cached webpages:
 identify a first physical location from which the first webpage was requested by the panelist device; and 
 transmit a third number of requests for the first webpage, the requests in the third number of requests identifying respective ones of a plurality of physical locations, the requests in the third number of requests skipping the first physical location from which the first webpage was requested by the panelist device, thereby reducing the total number of requests sent to the first webpage. 
   
     
     
         7 . The apparatus of  claim 1 , wherein the first number of requests for the first webpage are transmitted via a monitoring data collection site. 
     
     
         8 . An apparatus comprising:
 interface circuitry; and   processor circuitry to execute instructions to:
 identify a first webpage; 
 access a set of cached webpages cached in a panelist device; 
 when the first webpage is in the set of cached webpages:
 determine a first physical location from which the first webpage was requested by the panelist device; and 
 transmit a plurality of requests for the first webpage, the requests in the plurality of requests identifying respective ones of a plurality of physical locations, the requests in the plurality of requests skipping the first physical location from which the first webpage was requested by the panelist device, thereby reducing a number of requests sent to the first webpage. 
 
   
     
     
         9 . The apparatus of  claim 8 , wherein the set of cached webpages is accessed from a browser history of the panelist device. 
     
     
         10 . The apparatus of  claim 9 , wherein the browser history is native to a browser of the panelist device. 
     
     
         11 . The apparatus of  claim 8 , wherein the set of cached webpages is accessed from a log of webpages in the panelist device. 
     
     
         12 . The apparatus of  claim 8 , wherein the instructions, when executed, cause the processor circuitry to:
 when the first webpage is in the set of cached webpages:
 identify a first browser type used by the panelist device to access the first webpage; and 
 transmit a second plurality of requests for the first webpage, respective ones of the requests in the second plurality identifying different browser types, the requests in the second plurality of requests skipping the first browser type used by the panelist device to access the first webpage, thereby reducing a total number of requests sent to the first webpage. 
   
     
     
         13 . The apparatus of  claim 8 , wherein the instructions, when executed, cause the processor circuitry to:
 when the first webpage is in the set of cached webpages:
 identify a first language setting used by the panelist device when the first webpage was accessed; and 
 transmit a second plurality of requests for the first webpage, respective ones of the request in the second plurality identifying different language settings, the requests in the second plurality of requests skipping the first language setting used by the panelist device when the first webpage was accessed, thereby reducing the number of requests sent to the first webpage. 
   
     
     
         14 . The apparatus of  claim 8 , wherein the plurality of requests for the first webpage are transmitted via a monitoring data collection site. 
     
     
         15 . A method comprising:
 extracting, by executing an instruction with a processor, a language setting from a first web browser type of a panelist computing device; and   transmitting, by executing an instruction with a processor, data associated with a webpage accessed by the first web browser type and an indication of the language setting to audience measurement entity server, the audience measurement entity server to generate a number of web requests for the webpage from a web server hosting the webpage, the web requests indicative of a request to retrieve the webpage, respective ones of the web requests identifying different language settings than the extracted language setting, the respective ones of the web requests identifying different web browser types than the first web browser type.   
     
     
         16 . The method of  claim 15 , wherein the audience measurement entity server is to reduce a total number of requests sent to the web server in response to obtaining the indication of the language setting from the panelist computing device, the audience measurement entity server to generate the number of requests based on the indication of the language setting, respective ones of the requests identifying different language settings than the language setting used by the panelist computing device when the webpage was accessed. 
     
     
         17 . The method of  claim 15 , wherein the number of requests is a first number of requests, further including skipping the first web browser type used by the panelist computing device to access the webpage, thereby reducing a total number of requests sent to the webpage by the audience measurement entity server. 
     
     
         18 . The method of  claim 15 , wherein the number of requests is a first number of requests, further including:
 identifying a first physical location of the panelist computing device when the webpage was accessed; and   transmitting the first physical location to the audience measurement entity server, the audience measurement entity server to transmit a second number of requests for the webpage, respective ones of the requests in the second number identifying different physical locations than the first physical location, the requests in the second number of requests skipping the first physical location from which the webpage was requested by the panelist computing device, thereby reducing a total number of requests sent to the webpage by the audience measurement entity server.   
     
     
         19 . The method of  claim 18 , further including:
 storing the data in a data store by accessing a browser history of the panelist computing device, the browser history is native to the first web browser type of the panelist computing device.   
     
     
         20 . The method of  claim 19 , further including generating a log including the data in response to the first web browser type accessing the webpage, the log different from the browser history and a set of webpages cached in the panelist computing device.