System and Method for Automatically Detecting and Interactively Displaying Information About Entities, Activities, and Events from Multiple-Modality Natural Language Sources

Abstract

A method for automatically extracting and organizing information by a processing device from a plurality of data sources is provided. A natural language processing information extraction pipeline that includes an automatic detection of entities is applied to the data sources. Information about detected entities is identified by analyzing products of the natural language processing pipeline. Identified information is grouped into equivalence classes containing equivalent information. At least one displayable representation of the equivalence classes is created. An order in which the at least one displayable representation is displayed is computed. A combined representation of the equivalence classes that respects the order in which the displayable representation is displayed is produced.

What is claimed is: 
     
         1 . A method for automatically extracting and organizing information by a processing device from a plurality of data sources, comprising:
 applying to the data sources a natural language processing information extraction pipeline that includes an automatic detection of entities;   identifying information about detected entities by analyzing products of the natural language processing pipeline;   grouping identified information into equivalence classes containing equivalent information;   creating at least one displayable representation of the equivalence classes;   computing an order in which the at least one displayable representation is displayed; and   producing a combined representation of the equivalence classes that respects the order in which the displayable representation is displayed.   
     
     
         2 . The method of  claim 1 , wherein each equivalence class comprises a collection of items, each item comprising a span of text extracted from a document, together with a specification of information about a desired entity derived from the span of text. 
     
     
         3 . The method of  claim 1 , wherein computing an order in which the displayable representations are displayed further comprises randomly computing the order. 
     
     
         4 . The method of  claim 1 , wherein grouping identified information into equivalence classes further comprises assigning each identified information to a separate equivalence class. 
     
     
         5 . The method of  claim 1 , wherein grouping identified information into equivalence classes further comprises:
 computing a representative instance of each equivalence class;   ensuring that representative instances of different classes are not redundant with respect to each other;   ensuring that instances of each equivalence class are redundant with respect to the representative instance of the equivalence class.   
     
     
         6 . A method for processing information by a processing device, the method comprising:
 receiving a user query;   inferring a user query intention from the user query to develop an inferred user intention; and   automatically generating a page in response to the user query by adaptively building a template that corresponds to the inferred user intention using natural processing of multiple modalities comprising at least one of text, audio and video.   
     
     
         7 . The method of  claim 6 , further comprising: when the user query selects a person who has a political status,
 detecting the political status,   searching for information on at least one of an election campaign, public appearances, statements, and public service history, and   automatically generating a page in response to the user query.   
     
     
         8 . The method of  claim 6 , further comprising when the user query selects a company:
 searching for information on at least one of recent news about the company, information on the company's top officials, and press releases for the company; and   automatically generating a page in response to the user query.   
     
     
         9 . The method of  claim 6 , further comprising when the user query selects an event:
 searching for information on at least one of news items about the event and reactions to the event; and   automatically generating a page in response to the user query.   
     
     
         10 . The method of  claim 9 , wherein entities in the event are identified and retrieved relevant information about the entities is searched. 
     
     
         11 . A method for automatically extracting and organizing information by a processing device from a corpus of documents having multiple modalities of information in multiple languages for display to a user, the method comprising:
 browsing the corpus of documents to identify and incrementally retrieve documents containing audio/video files;   transcribing text from the audio/video files to provide a textual representation;   translating text of the textural representation that is in a foreign language;   incrementally extracting desired information about at least one of entities, activities, and events;   organizing extracted information; and   converting organized extracted information into a navigable display presentable to the user.   
     
     
         12 . The method of  claim 11 , wherein incrementally extracting desired information comprises:
 applying a natural language processing pipeline to each document to iterate all entities detected in the corpus;   identifying relation mentions and event mentions that involve a selected entity,   wherein an entity is at least one of a physical animate object, a physical inanimate object, something that has a proper name, something that has a measurable physical property, a legal entity and abstract concepts,   wherein a mention is a span of text that refers to an entity,   wherein a relation is a connection between two entities,   wherein a relation mention is a span of text that describes a relation, and   wherein an event is a set of relations between two or more entities involving one or more actions.   
     
     
         13 . The method of  claim 11 , wherein organizing extracted information comprises:
 iterating on all the entities identified in the corpus;   dividing the information extracted about the entity into selected equivalence classes containing equivalent information;   iterating on all the equivalence classes;   selecting one item in each equivalence class to represent all items in the equivalence class; and   recording information about the equivalence class and about a representative selected for use in producing the navigable display,   wherein each equivalence class comprises a collection of items, each item comprising a span of text extracted from a document, together with a specification of the information about the desired entity derived from the span of text.   
     
     
         14 . The method of  claim 11 , wherein converting organized extracted information into a navigable display presentable to the user comprises:
 scoring the equivalence classes of information by assigning to the equivalence class at least one of a highest score of the pieces of information in the class, the average score of its members, the median score of its members, and the sum of the scores of its members;   sorting the equivalence classes in descending order of score to prioritize an order in which the equivalence classes are displayed to the user;   iterating for each equivalence class a constructing of a displayable representation of an instance selected; and   combining the displayable representations to produce a displayable representation of the equivalence classes.   
     
     
         15 . The method of  claim 14 , wherein the displayable representation comprises a passage containing extracted information marked up with visual highlights.