Systems and Methods for Generating Summaries of Documents

Abstract

Systems and methods for summarizing online articles for consumption on a user device are disclosed herein. The system extracts the main body of an article's text from the HTML code of an online article. The system may then classify the extracted article into one of several different categories and removes duplicate articles. The system breaks down the article into its component sentences, and each sentence is classified into one of three categories: (1) potential candidate sentences that may be included in the generated summary; (2) weakly rejected sentences that will not be included in the summary but may be used to generate the summary; and (3) strongly rejected sentences that are not included in the summary. Finally, the system applies a document summarizer to generate quickly readable article summaries, for viewing on the user device, using relevant sentences from the article while maintaining the coherence of the article.

1 - 20 . (canceled) 
     
     
         21 . A system, comprising:
 an interface processor that provides an interactive graphical user interface to a user device over a network;   a parser processor that retrieves an RSS feed over the network, and generate an initial set of articles based on the RSS feed;   a categorization processor that categorizes each article in the initial set of articles into one of a plurality of subject matter categories;   a named entity recognizer that extracts entity mentions from the initial set of articles;   a named entity disambiguation module that identifies a named entity for each entity mention from at least one knowledge base;   a named entity salience identifier that identifies a salience for each named entity;   a deduplication processor that generates a final set of articles by removing duplicate articles from the initial set of articles, wherein each article in the final set of articles comprises a plurality of sentences and a title; and   a summarization processor that, for each article in the final set of articles:
 generates a preliminary score for each sentence in the plurality of sentences, 
 assigns each sentence in the plurality of sentences to one of three categories, 
 generates an article summary, wherein the article summary comprises one or more sentences from one of the three categories, wherein the article summary is based at least in part on the preliminary score for each sentence, and 
 provides the article summary to the user device over the network via the interface processor. 
   
     
     
         22 . The system of  claim 21 , wherein the named entity recognizer extracts entity mentions from the initial set of articles by identifying a plurality of entity boundaries in sentences in each of the initial set of articles and assigning a classification to each unique entity in the plurality of entity boundaries. 
     
     
         23 . The system of  claim 21 , wherein the named entities comprise at least one of a person, a place, and a thing.