Extracting and displaying compact and sorted results from queries over unstructured or semi-structured text

Abstract

A system for indexing unstructured or semi-structured data is disclosed. The system may identify regions within the data, such as “Abstract” or “References”. The system may identify linguistic units such as sentences, noun groups, verb groups. The system may also identify concepts such as companies, people, diseases, amounts, and so forth. The query results may be formatted so that similar results from different documents, or from the same document, are clustered together.

1 . A system comprising:
 a region detect component configured to analyze unstructured data and to identify one or more regions in the unstructured data;   a linguistic component configured to identify linguistic units in the unstructured data; and   an indexing component configured to store the unstructured data in an index based on the one or more identified regions and the one or more identified linguistic units.   
   
   
       2 . The system of  claim 1  wherein the region detect component is further configured to analyze semi-structured data that includes at least one region and to identify one or more additional regions in the semi-structured data. 
   
   
       3 . The system of  claim 1  wherein the region detect component is further configured to generate a semi-structured document comprising the unstructured data and the one or more identified regions. 
   
   
       4 . The system of  claim 1  wherein the linguistic component is further configured to identify regular and irregular morphological variants of words. 
   
   
       5 . The system of  claim 1  wherein the linguistic component is further configured to determine a position corresponding to each of the identified linguistic units, and wherein the indexing component is further configured to receive the determined position of each identified linguistic unit and record the position of the selected linguistic unit in the index. 
   
   
       6 . The system of  claim 1  wherein the identified linguistic units include concepts. 
   
   
       7 . The system of  claim 6  wherein the identified concepts are arranged in taxonomies. 
   
   
       8 . The system of  claim 6  wherein the linguistic component identifies the concepts based on a form of the concepts and a context in which the concepts appear in the unstructured data. 
   
   
       9 . The system of  claim 1  wherein the linguistic component is further configured to identify expressions. 
   
   
       10 . The system of  claim 9  wherein the identified expressions are selected from the group consisting of: numerical expressions, temporal expressions, phone numbers and email addresses. 
   
   
       11 . The system of  claim 9  wherein the linguistic component identifies the expressions based on a form of the expressions and a context in which the expressions appear in the unstructured data. 
   
   
       12 . The system of  claim 1  further comprising a query component configured to query the index based on one or more constraints. 
   
   
       13 . The system of  claim 12  wherein the one or more constraints are received from a user. 
   
   
       14 . The system of  claim 12  wherein the query component includes a drag and drop user interface. 
   
   
       15 . The system of  claim 12  wherein the one or more constraints include linguistic patterns. 
   
   
       16 . The system of  claim 12  further comprising a display component configured to format and display results of the query component to a user. 
   
   
       17 . The system of  claim 16  wherein the display component formats the results based on one or more rules. 
   
   
       18 . The system of  claim 17  wherein the one or more rules are configurable by a user. 
   
   
       19 . The method comprising:
 in response to receiving a query comprising a linguistic pattern,
 querying an index of unstructured text based on one or more constraints of the linguistic pattern, the index having been generated by identifying a plurality linguistic units in the unstructured text; and 
 formatting results of the query by clustering together similar results from a same document; and 
 clustering together similar results from different documents. 
   
   
   
       20 . The method of  claim 19  wherein the formatting is based on the linguistic pattern. 
   
   
       21 . The method of  claim 19  wherein the formatting is based on a plurality of rules having an order of precedence. 
   
   
       22 . The method of  claim 19  further comprising displaying the results to a user. 
   
   
       23 . The method of  claim 22  wherein the results are displayed in a format selected from the group consisting of: HTML, XLS, XML, CSV, and TSV. 
   
   
       24 . The method of  claim 22  further comprising providing one or more controls that enable the user to manipulate the displayed results. 
   
   
       25 . The method of  claim 24  wherein the one or more controls include an expand control to expand the displayed results. 
   
   
       26 . The method of  claim 22  wherein the results are displayed in alphabetical order. 
   
   
       27 . The method of  claim 22  wherein the results are displayed according to frequency order. 
   
   
       28 . The method of  claim 19  further comprising, in response to receiving a subsequent request to join the results to another linguistic pattern; and
 querying the index based on one or more constraints of the other linguistic pattern; and   joining the results to results of the other linguistic pattern query.   
   
   
       29 . A computer-readable storage medium comprising instructions that, when executed by a computer system, cause the computer system to:
 receive a query that includes at least one linguistic constraint and an indication of at least one region;   identify within a plurality of semi-structured and unstructured documents, at least one document that includes the at least one linguistic constraint within the at least one region of the document; and   format results of the query by clustering together similar results from the at least one document.   
   
   
       30 . The computer-readable storage medium of  claim 29  wherein the results are further formatted based on a plurality of rules having an order of precedence. 
   
   
       31 . The computer-readable storage medium of  claim 29  further comprising instructions that, when executed by the computer system, cause the computer system to display the results to a user. 
   
   
       32 . The computer-readable storage medium of  claim 31  wherein the results are displayed in a format selected from the group consisting of: HTML, XLS, XML, CSV, and TSV. 
   
   
       33 . The computer-readable storage medium of  claim 31  further comprising instructions that, when executed by the computer system, cause the computer system to provide one or more controls that enable the user to manipulate the displayed results. 
   
   
       34 . A system comprising:
 a region detect component configured to
 analyze unstructured data and to identify one or more regions in the unstructured data; and 
 analyze semi-structured data that includes at least one region and to identify one or more additional regions in the semi-structured data 
   a linguistic component configured to identify linguistic units in the unstructured and semi-structured data; and   a querying component configured to receive a query which identifies one or more linguistic units occurring within a region in combination with one or more linguistic patterns.   
   
   
       35 . The system of  claim 34  wherein the linguistic component is further configured to identify regular and irregular morphological variants of words. 
   
   
       36 . The system of  claim 34  wherein at least one of the identified linguistic units is a concept. 
   
   
       37 . The system of  claim 36  further comprising taxonomies, and wherein the taxonomies are used by the linguistic component to identify the at least one concept. 
   
   
       38 . The system of  claim 34  wherein the query is received from a user. 
   
   
       39 . The system of  claim 38  further comprising a display component configured to format and display results of the interactive query component to the user.