Systems and Methods for Identifying Terms Relevant to Web Pages Using Social Network Messages

Abstract

Systems and methods for retrieving social network messages and/or web pages in response to search queries are described. One embodiment of the invention includes generating a word list from at least a portion of the content of the web page using a web and message server system, generating an initial list of relevant terms based upon the word list using the web and message server system, identifying additional relevant terms using messages posted to at least one social network based upon the initial list of relevant terms, and creating an updated list of relevant terms by using the web and server system to combine terms in the initial list of relevant terms with the additional relevant terms identified using messages posted to at least one social network.

What is claimed: 
     
         1 . A method of identifying a list of terms relevant to a web page, comprising:
 generating a word list from at least a portion of the content of the web page using a web and message server system;   generating an initial list of relevant terms based upon the word list using the web and message server system;   identifying additional relevant terms using messages posted to at least one social network based upon the initial list of relevant terms; and   creating an updated list of relevant terms by using the web and server system to combine terms in the initial list of relevant terms with the additional relevant terms identified using messages posted to at least one social network.   
     
     
         2 . The method of  claim 1 , wherein generating a word list from at least a portion of the content of the web page using a web and message server system comprises:
 extracting desired content from the web page; and   generating a list of words utilized in the extracted web page content.   
     
     
         3 . The method of  claim 2 , wherein the desired content extracted from the web page includes content from the group consisting of the title, URL, links, and body of the web page. 
     
     
         4 . The method of  claim 2 , wherein extracting desired content from the web page comprises performing document object model analysis on the web page. 
     
     
         5 . The method of  claim 2 , wherein generating a list of words utilized in the extracted web page content comprises:
 generating a list of words that appear in the extracted web page content;   filtering the list of words to eliminate words identified in a predetermined list of stop words; and   filtering the list of words to remove case and tense variants of words.   
     
     
         6 . The method of  claim 1 , wherein generating an initial list of relevant terms based upon the word list using the web and message server system comprises:
 generating combinations of words that appear as neighboring words in the extracted web page content; and   combining the word combinations with the list of individual words to generate the initial list of relevant terms.   
     
     
         7 . The method of  claim 6 , wherein each of the generated combinations is limited to a predetermined number of words. 
     
     
         8 . The method of  claim 6 , further comprising scoring each term in the initial list of terms with respect to at least the extracted content from the web page. 
     
     
         9 . The method of  claim 8 , wherein scoring each of the terms with respect to at least the extracted content from the web page comprises:
 scoring each term based upon at least one characteristic including a characteristic from the group consisting of:
 the number of occurrences of the term in the extracted web page content; 
 the number of occurrences of the term in the original web page; 
 the uniqueness of the term; 
 the position of the term on the web page; and 
 combinations thereof. 
   
     
     
         10 . The method of  claim 9 , wherein uniqueness of a term is determined based upon the message rate of the term within at least one message stream. 
     
     
         11 . The method of  claim 10 , wherein:
 the uniqueness of a term increases below a predetermined threshold; and   the uniqueness of a term decreases above the predetermined threshold.   
     
     
         12 . The method of  claim 1 , wherein identifying additional relevant terms using messages posted to at least one social network based upon the initial list of relevant terms comprises:
 determining the uniqueness of all combinations of a predetermined selection of the highest scoring terms from the initial list of relevant terms; and   selecting combinations of the terms based upon the uniqueness of the combination.   
     
     
         13 . The method of  claim 12 , wherein uniqueness of a combination of terms is determined based upon the message rate of the combination of terms within at least one message stream. 
     
     
         14 . The method of  claim 13 , wherein:
 the uniqueness of the combination of terms increases below a predetermined threshold; and   the uniqueness of the combination of terms decreases above the predetermined threshold.   
     
     
         15 . The method of  claim 12 , wherein the predetermined selection of the highest scoring terms from the initial list of relevant terms is a predetermined number of the terms from the initial list with the highest scores. 
     
     
         16 . The method of  claim 12 , wherein the predetermined selection of the highest scoring terms from the initial list of relevant terms includes all terms from the initial list with scores exceeding a predetermined threshold. 
     
     
         17 . The method of  claim 12 , wherein creating an updated list of relevant terms by using the web and server system to combine terms in the initial list of relevant terms with the additional relevant terms identified using messages posted to at least one social network comprises:
 scoring each combination of terms with respect to at least the extracted content from the web page; and   adding the combinations of terms to the initial list of terms.   
     
     
         18 . The method of  claim 17 , further comprising:
 sorting the combinations of terms and the terms in the initial list of terms based upon score; and   selecting an updated list based upon a predetermined selection of the highest scoring terms from the sorted list.   
     
     
         19 . The method of  claim 17 , wherein scoring each combination of terms with respect to at least the extracted content from the web page comprises:
 scoring each combination of terms based upon at least one characteristic including a characteristic from the group consisting of:
 the number of occurrences of the term in the extracted web page content; 
 the number of occurrences of the term in the original web page; 
 the uniqueness of the term; 
 the position of the term on the web page; and 
 combinations thereof. 
   
     
     
         20 . A web and message server system, comprising:
 a processor; and   a memory connected to the processor, where the memory is configured to store a web and message application;   wherein the web and message application configures the processor to:
 generate a word list from at least a portion of the content of the web page; 
 generate an initial list of relevant terms based upon the word list; 
 identify additional relevant terms using messages posted to at least one social network based upon the initial list of relevant terms; and 
 create an updated list of relevant terms by combine terms in the initial list of relevant terms with the additional relevant terms identified based on messages posted to at least one social network.