Using cache and bloom filters for url lookups

Abstract

Enforcing a policy based at least in part on URL information is disclosed. A request to access a first uniform resource locator (URL) is received from a client device. A portion of the first URL, or a transformation thereof, is matched against a bloom filter. A first query is performed using the matched portion of the received first URL, and, in response to receiving a “no match” response to the first query, a second query that is different from the first query is performed. A policy is enforced based at least in part on a category received as a result of a second query.

What is claimed is: 
     
         1 . A system, comprising:
 a processor configured to:
 receive, from a client device, a request to access a first uniform resource locator (URL); 
 match at least a portion of the received first URL against a bloom filter; 
 in response to receiving an “accept” as a result of the match against the bloom filter, perform a first query using the matched portion of the received first URL, and in response to receiving a “no match” response to the first query, perform a second query that is different from the first query; and 
 based at least in part on a category received as a result of a second query, enforce a policy with respect to the request to access the received first URL; and 
   a memory coupled to the processor and configured to provide the processor with instructions.   
     
     
         2 . The system of  claim 1  wherein the content of the first and second query is the same, and wherein the first and second query are submitted to two different entities. 
     
     
         3 . The system of  claim 1  wherein the first query is performed against a cache associated with a database associated with the bloom filter. 
     
     
         4 . The system of  claim 1  wherein the first query is performed against a database associated with the bloom filter. 
     
     
         5 . The system of  claim 1  wherein the second query is a shortened version of the first query. 
     
     
         6 . The system of  claim 1  wherein the second query is performed against a database that is not associated with the bloom filter. 
     
     
         7 . The system of  claim 1  wherein the second query is performed against a cache that is not associated with a database associated with the bloom filter. 
     
     
         8 . The system of  claim 1  wherein at least one of the first query and the second query includes an MD5 hash of a representation of at least a portion of the first URL. 
     
     
         9 . The system of  claim 1  wherein the processor is further configured to rewrite the first URL. 
     
     
         10 . The system of  claim 9  wherein the processor is configured to match the rewritten URL against the bloom filter. 
     
     
         11 . The system of  claim 1  wherein the processor is further configured to associate the received category with the first URL in a cache. 
     
     
         12 . The system of  claim 1  wherein matching comprises matching a transformation of the first URL against the bloom filter. 
     
     
         13 . The system of  claim 12  wherein the transformation includes an MD5 hash. 
     
     
         14 . A method, comprising:
 receiving, from a client device, a request to access a first uniform resource locator (URL);   matching at least a portion of the received first URL against a bloom filter, using a processor;   in response to receiving an “accept” as a result of the match against the bloom filter, performing a first query using the matched portion of the received first URL, and in response to receiving a “no match” response to the first query, performing a second query that is different from the first query; and   based at least in part on a category received as a result of a second query, enforcing a policy with respect to the request to access the received first URL.   
     
     
         15 . The method of  claim 14  wherein the first query is performed against a database associated with the bloom filter. 
     
     
         16 . The method of  claim 14  wherein the second query is a shortened version of the first query. 
     
     
         17 . The method of  claim 14  wherein the second query is performed against a database that is not associated with the bloom filter. 
     
     
         18 . The method of  claim 14  further comprising rewriting the first URL. 
     
     
         19 . The method of  claim 14  further comprising associating the received category with the first URL in a cache. 
     
     
         20 . A computer program product embodied in a non-transitory computer readable storage medium and comprising computer instructions for:
 receiving, from a client device, a request to access a first uniform resource locator (URL);   matching at least a portion of the received first URL against a bloom filter;   in response to receiving an “accept” as a result of the match against the bloom filter, performing a first query using the matched portion of the received first URL, and in response to receiving a “no match” response to the first query, performing a second query that is different from the first query; and   based at least in part on a category received as a result of a second query, enforcing a policy with respect to the request to access the received first URL.