Methods and apparatus to throttle media access by web crawlers

Abstract

Disclosed examples include programmable circuitry to obtain source-identifying information from a media request message, the source-identifying information indicative of a media-requesting source associated with the media request message; determine a partner affiliation status of the media-requesting source based on a comparison of the source-identifying information with a source category identifier from a data collection facility, the partner affiliation status indicative of whether the media-requesting source has a partnership with a media provider to provide audience measurement information associated with the server system; and insert a delay tag in a media response message to the media-requesting source based on the partner affiliation status, the delay tag corresponding to a delay time period.

What is claimed is: 
     
         1 . A server system comprising:
 memory;   instructions in the memory; and   programmable circuitry to be programmed by the instructions to:
 obtain source-identifying information from a media request message, the source-identifying information indicative of a media-requesting source associated with the media request message; 
 determine a partner affiliation status of the media-requesting source based on a comparison of the source-identifying information with a source category identifier from a data collection facility, the partner affiliation status indicative of whether the media-requesting source has a partnership with a media provider to provide audience measurement information associated with the server system; and 
 insert a delay tag in a media response message to the media-requesting source based on the partner affiliation status, the delay tag corresponding to a delay time period. 
   
     
     
         2 . The server system as defined in  claim 1 , wherein the media-requesting source is a web crawler. 
     
     
         3 . The server system as defined in  claim 1 , wherein the programmable circuitry is to include a delay function in the delay tag, the delay function native to the media-requesting source. 
     
     
         4 . The server system as defined in  claim 1 , wherein the delay tag is executable by the media-requesting source to generate a time delay. 
     
     
         5 . The server system as defined in  claim 1 , wherein the programmable circuitry is to include a function in the delay tag, the function to cause the media-requesting source to perform an operation that causes a time delay before requesting data from a web site. 
     
     
         6 . The server system as defined in  claim 1 , wherein the programmable circuitry is to insert the delay tag in the media response message by selecting a pre-existing copy of media that includes the delay tag. 
     
     
         7 . The server system as defined in  claim 1 , wherein the media request message is either a hypertext transfer protocol GET request or a hypertext transfer protocol POST request. 
     
     
         8 . A system comprising:
 memory;   instructions in the memory; and   programmable circuitry to be programmed by the instructions to:
 obtain a media request message for media hosted by a server; 
 determine a source indicator of a media-requesting source associated with the media request message, the source indicator indicative of whether the media-requesting source is associated with a media provider to provide audience measurement information associated with the media provided by the media provider; and 
 include a delay tag in a media response message based on the source indicator, the delay tag to cause a delay time period based on the source indicator. 
   
     
     
         9 . The system as defined in  claim 8 , wherein the media-requesting source is a web crawler. 
     
     
         10 . The system as defined in  claim 8 , wherein the programmable circuitry is to include a delay function in the delay tag, the delay function native to the media-requesting source. 
     
     
         11 . The system as defined in  claim 8 , wherein the delay tag is executable by the media-requesting source to generate a time delay. 
     
     
         12 . The system as defined in  claim 8 , wherein the programmable circuitry is to include a function in the delay tag, the function to cause the media-requesting source to perform an operation that causes a time delay before requesting data from a web site. 
     
     
         13 . The system as defined in  claim 8 , wherein the programmable circuitry is to insert the delay tag in the media response message by selecting a pre-existing copy of the media that includes the delay tag. 
     
     
         14 . The system as defined in  claim 8 , wherein the media request message is either a hypertext transfer protocol GET request or a hypertext transfer protocol POST request. 
     
     
         15 . A system comprising:
 memory;   instructions in the memory; and   programmable circuitry to be programmed by the instructions to:
 identify source-identifying information corresponding to a sender of a media request message, the media request message corresponding to media hosted by a server; 
 associate the media request message with a label, the label to indicate capabilities of the sender; and 
 make a selection from (a) whether to enforce a server-side delay for access to the media and (b) whether to enforce a sender-side delay for access to the media, the selection based on the label associated with the media request message. 
   
     
     
         16 . The system as defined in  claim 15 , wherein the programmable circuitry is to identify whether the sender can execute executable instructions associated with the media. 
     
     
         17 . The system as defined in  claim 16 , wherein the programmable circuitry is to enforce the server-side delay for access to the media after a determination that the sender is unable to execute the executable instructions associated with the media. 
     
     
         18 . The system as defined in  claim 16 , wherein the programmable circuitry is to cause transmission of the executable instructions associated with the media to the sender, the executable instructions to specify a delay time period to occur prior to presenting the media. 
     
     
         19 . The system as defined in  claim 18 , wherein the programmable circuitry is to adjust the delay time period based on characteristics corresponding to the sender of the media request message. 
     
     
         20 . The system as defined in  claim 15 , wherein the programmable circuitry is to cause a delay time period prior to sending the media to the sender of the media request message.