Message compression in scalable messaging system

Abstract

A method comprises: retrieving encoded messages for a particular channel of a plurality of channels from respective buffers having time-to-lives that have not expired, wherein messages are encoded based on a pattern associated with each message for the particular channel; analyzing, by one or more computer processors, content of at least one retrieved encoded message; identifying, by the one or more computer processors, from the content the pattern used to encode each retrieved encoded message; decoding, by the one or more computer processors, each retrieved encoded message based on the identified pattern; and transmitting the decoded messages to one or more subscriber clients.

What is claimed is: 
     
         1 . A method, comprising:
 retrieving encoded messages for a particular channel of a plurality of channels from respective buffers having time-to-lives that have not expired,
 wherein messages are encoded based on a pattern associated with each message for the particular channel; 
   analyzing, by one or more computer processors, content of at least one retrieved encoded message;   identifying, by the one or more computer processors, from the content the pattern used to encode each retrieved encoded message;   decoding, by the one or more computer processors, each retrieved encoded message based on the identified pattern; and   transmitting the decoded messages to one or more subscriber clients.   
     
     
         2 . The method of  claim 1 , wherein retrieving encoded messages for the particular channel comprises:
 retrieving encoded messages from one or more of blocks within a first buffer having respective time-to-lives that have not expired.   
     
     
         3 . The method of  claim 1 , wherein the pattern is shared by at least some of the messages for the particular channel. 
     
     
         4 . The method of  claim 1 , comprising:
 receiving from a plurality of publisher clients a plurality of messages, each message being for a channel of the plurality of channels.   
     
     
         5 . The method of  claim 4 , wherein each channel comprises an ordered plurality of messages. 
     
     
         6 . The method of  claim 1 , comprising:
 encoding each message for the particular channel based on a dictionary for the particular channel,
 wherein the dictionary defines the pattern associated with each message for the particular channel. 
   
     
     
         7 . The method of  claim 6 , wherein encoding each message for the particular channel comprises:
 compressing the message for the particular channel according to the pattern.   
     
     
         8 . The method of  claim 6 , comprising:
 adding the identified pattern to the dictionary for the particular channel.   
     
     
         9 . The method of  claim 1 , comprising:
 storing the encoded messages for the particular channel in one or more respective buffers,   wherein each buffer comprises a respective time-to-live and resides on a respective node.   
     
     
         10 . The method of  claim 1 , wherein decoding each retrieved encoded message comprises:
 decompressing each message in the particular channel according to the identified pattern.   
     
     
         11 . A system, comprising:
 one or more computer processors programmed to perform operations to:
 retrieve encoded messages for a particular channel of a plurality of channels from respective buffers having time-to-lives that have not expired,
 wherein messages are encoded based on a pattern associated with each message for the particular channel; 
 
 analyze content of at least one retrieved encoded message; 
 identify from the content the pattern used to encode each retrieved encoded message; 
 decode each retrieved encoded message based on the identified pattern; and 
 transmit the decoded messages to one or more subscriber clients. 
   
     
     
         12 . The system of  claim 11 , wherein the pattern is shared by at least some of the messages for the particular channel. 
     
     
         13 . The system of  claim 11 , wherein the operations are further to:
 receive from a plurality of publisher clients a plurality of messages, each message being for a channel of the plurality of channels.   
     
     
         14 . The system of  claim 13 , wherein each channel comprises an ordered plurality of messages. 
     
     
         15 . The system of  claim 11 , wherein the operations are further to:
 encode each message for the particular channel based on a dictionary for the particular channel,
 wherein the dictionary defines the pattern associated with each message for the particular channel. 
   
     
     
         16 . The system of  claim 15 , wherein to encode each message for the particular channel the one or more computer processors are further to:
 compress the message for the particular channel according to the pattern.   
     
     
         17 . The system of  claim 15 , wherein the operations are further to:
 add the identified pattern to the dictionary for the particular channel.   
     
     
         18 . The system of  claim 11 , wherein the operations are further to:
 store the encoded messages for the particular channel in one or more respective buffers,
 wherein each buffer comprises a respective time-to-live and resides on a respective node. 
   
     
     
         19 . The system of  claim 11 , wherein to decode each retrieved encoded message the one or more computer processors are further to:
 decompress each message in the particular channel according to the identified pattern.   
     
     
         20 . A non-transitory computer-readable medium having instructions stored thereon that, when executed by one or more computer processors, cause the one or more computer processors to:
 retrieve encoded messages for a particular channel of a plurality of channels from respective buffers having time-to-lives that have not expired,
 wherein messages are encoded based on a pattern associated with each message for the particular channel; 
   analyze content of at least one retrieved encoded message;   identify from the content the pattern used to encode each retrieved encoded message;   decode each retrieved encoded message based on the identified pattern; and   transmit the decoded messages to one or more subscriber clients.