Extracting information from tables embedded within documents

Abstract

Much valuable information in documents is presented within tables. However, the information within tables is hard to extract automatically with high accuracy due to the wide variety and low quality of typical tables found in electronic documents. Information extraction technology can provide a method of extracting information from heterogeneous tables by recognizing tables, the header cells, and cells that are merged or should be merged, creating a richer representation of table structure and providing a convenient way of linking cells to their row and column headers. Use of this richer representation allows a few extraction patterns to successfully pull out information from a wide variety of differently formatted tables.

We claim: 
     
         1 . A computing device implemented methodof extracting information from heterogeneous tables in semi-structured text and unstructured text, the method comprising steps of:
 identifying, by a computing device, target content from a table in an electronic document, wherein the target content is presented in a plurality of cells table cell context within a document;   classifying, by the computing device, each table cell as the plurality of cells into one or more of a header cells and a plurality of cell or data cells cell based on at least one of explicit coding of the plurality of cells, formatting of the plurality of cells, relationship between the one or more header cells and columns in the table, presence of horizontal lines in the table, type of the target content in the plurality of cells, presence of measurement units within brackets in the table, and presence of words referring to mathematical operations on values in a table its context or content;   annotatingdirectly encoding, automatically by the computing device, the plurality of data cells cell with annotations to indicate their positions the data cell's position in the a table and an association between each of the plurality of data cells cell and the one or more header cells cell to enable extraction of the target content from the table; and   indexing, by the computing device, the electronic document utilizing the association between the plurality of data cells cell and the one or more header cells for responding to search queries cell.   
     
     
         2 . The computing device implemented method of  claim 1 , wherein the target content corresponds to semi-structured text that does not explicitly provide relationships between the plurality of cells and the one or more headers cellsnumeric identifiers are used to identify a position of each cell in the table within the document. 
     
     
         3 . The computing device implemented method of claim  2   1 , wherein the electronic document is selected from one of HTML and XML documents and includes format tagsfurther comprising:
 merging two or more cells to correct one or more table structures in the document.   
     
     
         4 . The computing device implemented method of  claim 1 , wherein the target content corresponds to plain text and the step of identifying the target content identifies at least one of lines within text, multiple rows where text or white space is aligned, and table captions or headers. 
     
     
         5 . The computing device implemented method of  claim 1 , further comprising a step of converting the target content into semi-structured text wherein the header cell is also identified by formatting differences between the header cell and other cells in the document.   
     
     
         6 . The computing device implemented method of claim  4   1 , wherein the step of converting comprisesfurther comprising steps of:
 establishing a number of columns based on differences in an amount of white space between two columns of text; and   recognizing columns that the table captions or header spans, or establishing cell contents according to alignment of contents and white space.   
     
     
         7 . The computing device implemented method of  claim 1 , further comprising a step of classifying the one or more of header cells into one or more column header cells and one or more row header cells based at least partially on a position of the one or more header cells in the table:
 indexing different formats of the document in an identical way to allow for more searches to be performed in the document.   
     
     
         8 . The computing device implemented method of claim  7   1 , wherein the step of annotating associates each of the plurality of data cell with the one or more column header cells or the one or more row header cells cell is also identified by one or more words referring to operations. 
     
     
         9 . A non-transitory computer-readable medium storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform a method of extracting information from heterogeneous tables in semi-structured text and unstructured text, the method comprising steps ofA computer program product comprising a tangible storage medium encoded with processor-readable instructions that, when executed by one or more processors, enable the computer program product to:
 identifying, by a computing device, target content from a table in an electronic document, wherein the target content is presented in a plurality of cellsidentify table cell context within a document;   classifying, by the computing device, the plurality of cells into one or more of header cells and a plurality of data cellsclassify each table cell as a header cell or data cell based on its context or content;   annotatingdirectly encode, automatically by the computing deviceone or more processors, the plurality of data cells cell with annotations to indicate their positions the data cell's position in the a table, and an association between each of the plurality of data cells cell and the one or more header cells cell to enable extraction of the target content from the table; and   extractingextract, by the computing devicecomputer program product, the target content from the table utilizing the association between plurality ofthe data cellscell and one or more header cells for the target content extraction requests cell.   
     
     
         10 . The non-transitory computer-readable mediumcomputer program product of  claim 9 , wherein the electronic document is selected from one of HTML and XML documents and includes at least one of semi-structured text and unstructured textheader cell is identified by one or more measurement units within the header cell. 
     
     
         11 . The non-transitory computer-readable mediumcomputer program product of  claim 9 , wherein the step of classifying is based on at least one of explicit coding of the plurality of cells, formatting of the plurality of cells, relationship between the one or more header cells and columns in the table, presence of horizontal lines in the table, type of the target content in the plurality of cells, presence of measurement units within brackets the table, and presence of words referring to mathematical operations on values in a table. 
     
     
         12 . The non-transitory computer-readable mediumcomputer program product of  claim 9 , wherein the step of annotating indicates at least one of target content and position of the header cellstwo or more of the cells within the table are merged. 
     
     
         13 . The non-transitory computer-readable medium of  claim 9 , wherein the step of annotating indicates target content of the one or more header cells and a position of the one or more header cells. 
     
     
         14 . The non-transitory computer-readable mediumcomputer program product of  claim 9 , further comprising a step of generating a representation of the table utilizing the indications of the one or more header cellswherein structures within the table are corrected by merging cells and/or rows. 
     
     
         15 . The non-transitory computer-readable mediumcomputer program product of  claim 9 , further comprising a step of identifying one or more of the plurality of cells that span multiple columns or rowswherein the document is optimized for text mining. 
     
     
         16 . The non-transitory computer-readable mediumcomputer program product of claim  15 , further comprising a step of expanding the identified one or more cells9, wherein a format of the document is optimized to increase efficiency during one or more searches. 
     
     
         17 . A computer system of extracting information from heterogeneous tables in semi-structured text and unstructured textconnected to a network, the system comprising:
 one or more processors configured to:   identify, target content from a table in an electronic document, wherein the target content is presented in a plurality of cells table cell context within a document;   classify the plurality of cells into one or more row- or column-defining cells and a plurality of data cells table cell context as a header cell or data cell based on its context or content;   automatically annotate directly encode the plurality of data cell with annotations to indicate their positions the data cell's position in the a table and an association between each of the plurality of the data cells cell and the one or more row- or column-defining cells header cell to enable extraction of the target content from the table; and   generate a representation of a the table based at least partially on the association of each of between the plurality of data cells cell with one or more row- or column-defining defining cells and the header cell.   
     
     
         18 . The computer system of  claim 17 , wherein the one or more processors are further configured to classify the one or more row- or column-defining cells into a subset of column header cells and a subset of row header cells based at least partially on a position of the one or more row- or column-defining cells in the tablerows having a substantially similar structure are merged after the header cell and data cell have been annotated. 
     
     
         19 . The computer system of claim  18   17 , wherein  at least one of the one or more row- or column-defining cell is classified as both a column header cell and a row header cell the document is converted into different formats to increase efficiency in searching for information within the table. 
     
     
         20 . The computer system of claim  18   17 , wherein  the one or more processors are further configured to merge two or more of the plurality of data cells two or more other cells having unmatched brackets are merged within the table. 
     
     
         21 . The system of  claim 20 , wherein the two or more cells are merged based on at least one of the plurality of data cells within a proximity or an alignment of text being empty. 
     
     
       22. The computing device implemented method of  claim 1 , wherein the annotations to indicate the data cell's position in the table comprise identifiers reflecting a row number and a column number of the table occupied by the data cell.