System and method for detecting unwanted content

Abstract

A system and method for detecting unwanted electronic content, such as spam. As a user operates an application to send messages, several metrics are tracked to allow the system to analyze her activity. Illustrative metrics may include, but are not limited to, block count (e.g., how many other users have blocked her), block rate (e.g., the rate at which other users block her), peer symmetry (e.g., percentage of her messages that are to other users that have her in their address book), message uniqueness (e.g., how unique her messages are from each other), peer uniqueness (e.g., how unique the addressees of her messages are), and message rate (e.g., the rate at which she sends messages). Periodically, metrics may be compared to corresponding thresholds. Depending on whether a threshold is crossed, and which threshold is crossed, she may be banned from using the application or placed on a watch list.

What is claimed is: 
     
       1. A method comprising:
 receiving, at a server from a first user, an electronic message having one or more addressees; 
 comparing a representation of the electronic message to representations of one or more electronic messages previously sent by the first user during a predetermined time period; 
 if the electronic message was previously sent by the first user during the predetermined time period, incrementing a message count associated with the electronic message; 
 if the electronic message was not previously sent by the first user during the predetermined time period, storing the representation of the electronic message; 
 updating a message uniqueness metric in a manner depending on whether the electronic message had been sent previously during the predetermined time period; 
 wherein the message uniqueness metric reflects a percentage of electronic messages sent by the first user during the predetermined time period that were unique; 
 comparing the updated message uniqueness metric to a threshold message uniqueness value indicative of distribution of unwanted content; 
 determining, based on the comparing of the updated message uniqueness metric, whether the electronic message contains unwanted content; 
 delivering the electronic message if the updated message uniqueness metric indicates that the electronic message does not contain unwanted content; and 
 refraining from delivering the electronic message if the updated message uniqueness metric indicates that the electronic message contains unwanted content. 
 
     
     
       2. The method of  claim 1 , further comprising:
 for each addressee in the one or more addressees, updating a peer symmetry metric to reflect whether the first user is included in an address book of the addressee; 
 updating a message rate reflecting a rate at which the first user is sending electronic messages; and 
 comparing the updated message rate to one or more threshold message rates. 
 
     
     
       3. The method of  claim 2 , further comprising:
 adding the first user to an address book of a second user; and 
 updating the peer symmetry metric. 
 
     
     
       4. The method of  claim 2 , further comprising:
 if the updated message rate exceeds a first message rate threshold, preventing the user from sending additional electronic messages; and 
 if the updated message rate exceeds a second message rate threshold, but not the first message rate threshold, add the first user to a watch list. 
 
     
     
       5. The method of  claim 4 , further comprising, if the user is placed in the watch list:
 initiating tracking of a peer uniqueness metric identifying a percentage of addressees of electronic messages sent by the first user that are unique. 
 
     
     
       6. The method of  claim 5 , wherein tracking the peer uniqueness metric comprises:
 comparing a first addressee of the electronic message to addressees of one or more previous electronic messages sent by the first user during a predetermined time period; 
 if the first addressee was previously sent an electronic message from the first user during the predetermined time period, incrementing a message count associated with the first addressee; 
 if the first addressee was not previously sent an electronic message from the first user during the predetermined time period, recording the first addressee; and 
 updating the peer uniqueness metric depending on whether the first addressee previously had been sent a message by the first user during the predetermined time period. 
 
     
     
       7. The method of  claim 1 , further comprising:
 receiving from a second user a block request regarding the first user; 
 incrementing a total block count of the first user; 
 updating a block rate reflecting a rate at which the first user is blocked by other users; and 
 adding the first user to the watch list if the block rate exceeds a first block rate threshold. 
 
     
     
       8. The method of  claim 1 , further comprising, prior to said receiving from the first user an electronic message:
 logging the first user into a computer system hosting an application for sending electronic messages; 
 retrieving from storage a first set of metrics, including a total block count reflecting how many other users have blocked the first user; 
 if the user is included in a watch list, retrieving from storage a second set of metrics, including: 
 the message uniqueness metric; and 
 a peer uniqueness metric identifying a percentage of addressees of electronic messages sent by the first user that are unique; and 
 if the user is not identified on the watch list, initializing the second set of metrics. 
 
     
     
       9. The method of  claim 8 , further comprising:
 retrieving a first list of other users to whom the first user is subscribed; 
 retrieving a second list of other users that have blocked the first user; and 
 striking from the first list any users identified in the second list. 
 
     
     
       10. The method of  claim 1 , further comprising:
 logging the user out of a computer system hosting an application for sending electronic messages; 
 storing a first set of metrics, including a total block count reflecting how many other users have blocked the first user; and 
 if the user is included in a watch list, storing a second set of metrics, including: the message uniqueness metric; and 
 a peer uniqueness metric identifying a percentage of addressees of electronic messages sent by the first user that are unique. 
 
     
     
       11. A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform a method comprising:
 receiving, at a server from a first user, an electronic message having one or more addressees; 
 comparing a representation of the electronic message to representations of one or more electronic messages previously sent by the first user during a predetermined time period; 
 if the electronic message was previously sent by the first user during the predetermined time period, incrementing a message count associated with the electronic message; 
 if the electronic message was not previously sent by the first user during the predetermined time period, storing the representation of the electronic message; 
 updating a message uniqueness metric in a manner depending on whether the electronic message had been sent previously during the predetermined time period; 
 wherein the message uniqueness metric reflects a percentage of electronic messages sent by the first user during the predetermined time period that were unique; 
 comparing the updated message uniqueness metric to a threshold message uniqueness value indicative of distribution of unwanted content; 
 determining, based on the comparing of the updated message uniqueness metric, whether the electronic message contains unwanted content; 
 delivering the electronic message if the updated message uniqueness metric indicates that the electronic message does not contain unwanted content; and 
 refraining from delivering the electronic message if the peer symmetry metric or the comparison indicates that the electronic message contains unwanted content. 
 
     
     
       12. An apparatus comprising:
 one or more processors; and 
 a non-transitory computer-readable medium storing instructions that, when executed by the one or more processors, cause the apparatus to:
 receive, at a server from a first user, an electronic message having one or more addressees; 
 compare a representation of the electronic message to representations of one or more electronic messages previously sent by the first user during a predetermined time period; 
 if the electronic message was previously sent by the first user during the predetermined time period, increment a message count associated with the electronic message; 
 if the electronic message was not previously sent by the first user during the predetermined time period, store the representation of the electronic message; 
 update a message uniqueness metric in a manner depending on whether the electronic message had been sent previously during the predetermined time period; 
 compare the updated message uniqueness metric to a threshold message uniqueness value indicative of distribution of unwanted content; 
 determine, based on the comparing of the updated message uniqueness metric, whether the electronic message contains unwanted content; 
 deliver the electronic message if the updated message uniqueness metric indicates that the electronic message does not contain unwanted content; and 
 refrain from delivering the electronic message if the updated message uniqueness metric indicates that the electronic message contains unwanted content. 
 
 
     
     
       13. The apparatus of  claim 12 , the non-transitory computer-readable storage medium further storing instructions that, when executed by the one or more processors, cause the apparatus to:
 for each addressee in the one or more addressees, update a peer symmetry metric to reflect whether the first user is included in an address book of the addressee; 
 update a message rate reflecting a rate at which the first user is sending electronic messages; and 
 compare the updated message rate to one or more threshold message rates. 
 
     
     
       14. The apparatus of  claim 13 , the non-transitory computer-readable storage medium further storing instructions that, when executed by the one or more processors, cause the apparatus to:
 add the first user to an address book of a second user; and 
 update the peer symmetry metric. 
 
     
     
       15. The apparatus of  claim 13 , the non-transitory computer-readable storage medium further storing instructions that, when executed by the one or more processors, cause the apparatus to:
 if the updated message rate exceeds a first message rate threshold, prevent the user from sending additional electronic messages; and 
 if the updated message rate exceeds a second message rate threshold, but not the first message rate threshold, add the first user to a watch list. 
 
     
     
       16. The apparatus of  claim 15 , the non-transitory computer-readable storage medium further storing instructions that, when executed by the one or more processors, cause the apparatus to:
 if the user is placed in the watch list, initiate tracking of a peer uniqueness metric identifying a percentage of addressees of electronic messages sent by the first user that are unique. 
 
     
     
       17. The apparatus of  claim 16 , wherein tracking the peer uniqueness metric comprises:
 comparing a first addressee of the electronic message to addressees of one or more previous electronic messages sent by the first user during a predetermined time period; 
 if the first addressee was previously sent an electronic message from the first user during the predetermined time period, incrementing a message count associated with the first addressee; 
 if the first addressee was not previously sent an electronic message from the first user during the predetermined time period, recording the first addressee; and 
 updating the peer uniqueness metric depending on whether the first addressee previously had been sent a message by the first user during the predetermined time period. 
 
     
     
       18. The apparatus of  claim 12 , the non-transitory computer-readable storage medium further storing instructions that, when executed by the one or more processors, cause the apparatus to:
 receive from a second user a block request regarding the first user; 
 increment a total block count of the first user; 
 update a block rate reflecting a rate at which the first user is blocked by other users; and 
 add the first user to the watch list if the block rate exceeds a first block rate threshold. 
 
     
     
       19. The apparatus of  claim 12 , the non-transitory computer-readable storage medium further storing instructions that, when executed by the one or more processors, cause the apparatus to, prior to said receiving from the first user an electronic message:
 log the first user into a computer system hosting an application for sending electronic messages; 
 retrieve from storage a first set of metrics, including a total block count reflecting how many other users have blocked the first user; 
 if the user is included in a watch list, retrieve from storage a second set of metrics, including:
 the message uniqueness metric; and 
 a peer uniqueness metric identifying a percentage of addressees of electronic messages sent by the first user that are unique; and 
 if the user is not identified on the watch list, initialize the second set of metrics. 
 
 
     
     
       20. The apparatus of  claim 19 , the non-transitory computer-readable storage medium further storing instructions that, when executed by the one or more processors, cause the apparatus to:
 retrieve a first list of other users to whom the first user is subscribed; 
 retrieve a second list of other users that have blocked the first user; and 
 strike from the first list any users identified in the second list. 
 
     
     
       21. The apparatus of  claim 12 , the non-transitory computer-readable storage medium further storing instructions that, when executed by the one or more processors, cause the apparatus to:
 log the user out of a computer system hosting an application for sending electronic messages; 
 store a first set of metrics, including a total block count reflecting how many other users have blocked the first user; and 
 if the user is included in a watch list, store a second set of metrics, including:
 the message uniqueness metric; and 
 a peer uniqueness metric identifying a percentage of addressees of electronic messages sent by the first user that are unique.