Apparatus and Method for Conducting Searches with a Search Engine for Unstructured Data to Retrieve Records Enriched with Structured Data and Generate Reports Based Thereon

Abstract

Records in databases or unstructured files are enriched with metadata and are indexed for retrieval by a search engine. In response to a search request, a graphical user interface (GUI) control based on the metadata associated with the search hits is constructed and displayed with the search results in a standard view. Selection of a metadata value via the GUI control filters the previously matched records down to those matching the value selected via the GUI control. The metadata in the search results is arranged in a tabular view which is embedded in the display of search results and rendered invisible until selected by the user. Reports can be constructed from an identifier each returned record set for presenting, analyzing and modifying the data, and for generating further reports.

1 . A method of winnowing down a list of search results, identifying records in one or more data sources, returned by a search for matches between a search criterion and an index of a population of records stored in machine readable form, comprising
 a. enriching each of said records with metadata comprising the name of a tag and at least one value of for said tag, each name of a tag and corresponding value forming a tag-value pair,   b. comparing the content of the index with said search criterion,   c. displaying said list of search results which satisfy said search criterion,   d. displaying tag names and corresponding tag values in the metadata of the records and, until said search is to be discontinued, performing a further iteration of said search by   e. selecting a tag value,   f. modifying said search criterion as a function of the selected tag value, and   g. repeating steps b, c and d.   
   
   
       2 . A method according to  claim 1  wherein step c further comprises displaying said list of search results in a standard view, step d. further comprises displaying a GUI control labeled with said tag names and said corresponding tag values in the metadata of the records, and step e. further comprises selecting a tag value on said GUI control. 
   
   
       3 . A method according to  claim 1  wherein said enriching each of said records further comprises integration of disparate data sources or applications which do not reside in said data source. 
   
   
       4 . A method according to  claim 1  comprising generating a URL for each search result. 
   
   
       5 . A method according to  claim 1  wherein said URL is modified as a function of the selected tag value. 
   
   
       6 . A method according to  claim 4  further comprising constructing a data query as a function of said URL,
 applying said query to a database for obtaining an answer set of records satisfying said query, and   generating a report as a function of information contained in said answer set of records.   
   
   
       7 . A method according to  claim 5  further comprising adding to said URL the identity of an external set of records not included in said list of records for including in said report information not contained in said list of records. 
   
   
       8 . A method according to  claim 1  further comprising, for each displayed tag name assigned to one field in each of two or more of said data sources, merging the lists values for said field in each of said data sources and displaying said list of values under a single tag name. 
   
   
       9 . A method according to  claim 1  further comprising, for each displayed tag name, counting the number of records having metadata including said tag name and displaying said number in association with said tag name. 
   
   
       10 . A method according to  claim 9  wherein said records are distributed among two or more of said data sources and the same tag name is used to identify one field in each of said data sources. 
   
   
       11 . A method according to  claim 1  further comprising, for each displayed tag value, counting the number of records having metadata including said tag value and displaying said number in association with said tag value. 
   
   
       12 . A method according to  claim 4  further comprising storing each URL before it is modified for enabling the list of the records which satisfy each search to be accessed. 
   
   
       13 . A method according to  claim 1  further comprising displaying in a tabular view of said search results a search result table having columns of cells identified by the tag names in the metadata of said list of records, each cell displaying a value for the tag name that identifies its column. 
   
   
       14 . A method according to  claim 13  further comprising displaying in said tabular view of said search results the title of each search result. 
   
   
       15 . A method according to  claim 13  further comprising displaying with each of the search results of said list at least one link to a corresponding meta query,
 applying each meta query selected by actuating its corresponding link to a database, and   preparing a report which is a function of the records in said database returned by said actuated meta query.   
   
   
       16 . A method according to  claim 13  wherein step e. further comprises selecting a tag value in a cell in said search result table. 
   
   
       17 . A method according to  claim 13  further comprising
 rendering said tabular view invisible when said standard view is visible,   rendering said standard view invisible when said tabular view is visible,   and providing a control for selectively rendering one of said standard view and said tabular view visible.   
   
   
       18 . A method according to  claim 13  further comprising displaying with said search result table at least one further GUI control for operating on said search result table and/or the data contained therein. 
   
   
       19 . A method according to  claim 18  wherein said one further GUI control enables the generation of a filter to be applied against the data in said search result table for filtering said data without querying said index. 
   
   
       20 . A method according to  claim 18  wherein said one further GUI control enables the generation of query to said search result table to transform the form of the data in said search result table to that of a chart. 
   
   
       21 . A method according to  claim 18  wherein said one further GUI control enables the generation of query to said table to perform calculations on the data in said search result table. 
   
   
       22 . A method according to  claim 18  wherein said one further GUI control enables the generation of query to said search result table to display a ranking of the data in said search result table. 
   
   
       23 . A method according to  claim 18  wherein said one further GUI control enables the generation of query to said search result table to save said search result table. 
   
   
       24 . A method according to  claim 18  wherein said one further GUI control enables the generation of query to said search result table to save a modified table derived from in said search result table. 
   
   
       25 . A method according to  claim 18  wherein said one further GUI control enables the generation of query to said search result table to email said search result table. 
   
   
       26 . A method according to  claim 18  wherein said one further GUI control enables the generation of query to said search result table to email a modified table derived from said search result table. 
   
   
       27 . A method according to  claim 18  wherein said one further GUI control enables the generation of query to export said search result table to another application. 
   
   
       28 . A method according to  claim 18  wherein said one further GUI control enables the generation of query to export a modified table derived from said search result table to another application. 
   
   
       29 . A method according to  claim 1  wherein said metadata comprises a hierarchy of tag names, each lowest level tag name being paired with at least one value, and each tag name having a higher level than the lowest level identifying a category of which said each immediately lower level tag name is a member. 
   
   
       30 . A method according to  claim 1  further comprising displaying with each of the search results of said list a link to at least one meta query,
 applying each meta query selected by actuating its corresponding link to a database, and   preparing a report which is a function of the records in said database returned by said actuated meta query.   
   
   
       31 . Method according to  claim 12  further comprising
 generating a meta query as a function of said stored URL,   applying said meta query to a database, and   preparing a report which is a function of the records in said database returned by said meta query.   
   
   
       32 . A system for searching in an index of a population of records for a subset of said records, each record of said subset having indexed content which matches the content of a search request, comprising,
 one or more data storage components for storing said population of records,   a data enrichment component operatively connected to said data storage component for enriching at least some of the records of said population with metadata indicative of a least one tag name and at least one value for a tag identified by said tag name,   an indexing component operatively connected to said data enrichment component for summarizing the content of said population of records in an index,   a search request input component for receiving said search request,   a search engine component operatively connected to said indexing component and said search request input component for returning from said index identifiers for said subset of records from said population of records,   a user interface component operatively connected to said search engine and including a display component for presenting a list of said identifiers,   a GUI control construction component for presenting on said user display component a GUI control adapted to display the tag names and values in the metadata of the records of said subset,   said GUI control construction component being operatively connected to said search engine component for, in response to selection of a displayed tag value on said GUI control, causing said search engine component to select a new subset of records from a population of records including the last subset of records, said user display component presenting a list of identifiers for the respective records in said new subset, and said GUI control construction component presenting on said user display component a GUI control adapted to display the tag names and values in the metadata of the records of said new subset which values are selectable to further cause said search engine component to select a new subset of records from a population of records including the last subset of records.   
   
   
       33 . A system according to  claim 32  further comprising a record transformer operatively connected between said indexing component and said data enrichment component for formatting said records for indexing. 
   
   
       34 . Apparatus according to  claim 32  further comprising a listener operatively connected between said data storage component and said data enrichment component for determining each event in which a new record is added to said data storage component or a record existing in said data storage component is modified, and
 in response to said determination, submitting said added or modified record to said data enrichment component.   
   
   
       35 . Apparatus according to  claim 32  further comprising a dynamic tabular view construction component for presenting on said user display component a tabular view of the tag names and values in the metadata of the records of each subset of records. 
   
   
       36 . Apparatus according to  claim 32  further comprising a counter for counting the number of records having a value which is displayed on said GUI control, said user display component being operatively connected to said counter for displaying next to each value and/or tag name the number of records which include said value and/or tag name in its metadata. 
   
   
       37 . Apparatus according to  claim 36  further comprising search result storage means for storing a reference to each subset of records returned by an iteration of said search, said search result storage means being operatively connected to said search engine for selectively causing said search engine to return an identifier for each subset of records. 
   
   
       38 . Apparatus according to  claim 37  wherein said identifier comprises a URL.