Automatic forms processing systems and methods

Abstract

Systems and methods analyze the physical structure of text rows in a document image, including the positions of one or more alignments of one or more character blocks in one or more text rows of the document image. The systems and methods determine one or more groups of text rows that are placed into a class based on the structures of the text rows, such as the positions of the one or more alignments of the one or more character blocks in each text row.

1 . A system to process at least one document image comprising a plurality of text rows and a plurality of characters, each text row having at least one character, the system comprising:
 at least one processor; and   a plurality of modules to execute on the at least one processor, the modules comprising:
 a character block creator to create character blocks for the characters in the text rows and to determine positions of alignments of the character blocks; and 
 a classification system to determine columns for the alignments of the character blocks at the positions of the alignments, each text row having a physical structure defined by the columns of the alignments of the character blocks in that text row, and to determine classes for the text rows based on the physical structures of the text rows as defined by the columns of the character blocks in each text row. 
   
     
     
         2 . The system of  claim 1  wherein the classification system determines the classes by:
 determining an initial subset of rows for each column having more than one character block aligned in that column in the text rows, each initial subset of rows comprising one or more text rows having at least one alignment of at least one character block in a selected column, each initial subset of rows having a set of columns comprising the selected column and other columns in the one or more text rows therein;   determining an optimum set for each initial subset of rows, each optimum set comprising a most representative set of columns selected from the set of columns of one of the initial subsets of rows;   determining a final subset of rows for each initial subset of rows, each final subset of rows comprising at least some of the one or more text rows of a corresponding initial subset of rows that have physical structures that are most similar to a corresponding most representative set of columns of a corresponding optimum set when compared to physical structures of all of the one or more text rows of the corresponding initial subset of rows;   determining a confidence factor for each final subset of rows, each confidence factor measuring a similarity of the physical structures of each one of the at least some text rows in one corresponding final subset of rows to each other one of the at least some text rows in the one corresponding final subset of rows;   determining a best confidence factor for each particular text row in the at least one document image, each particular text row having one or more confidence factors corresponding to one or more final subsets of rows in which the particular text row is an element; and   creating one or more classes of text rows, each class comprising one or more particular text rows having a same best confidence factor.   
     
     
         3 . The system of  claim 1  wherein the classification system comprises:
 a subsets module to determine the columns for the alignments of the character blocks and to determine an initial subset of rows for each column having more than one character block aligned in that column in the text rows, each initial subset of rows comprising one or more text rows having at least one alignment of at least one character block in a selected column, each initial subset of rows having a set of columns comprising the selected column and other columns in the one or more text rows therein;   an optimum set module to determine an optimum set for each initial subset of rows, each optimum set comprising a most representative set of columns selected from the set of columns of a corresponding initial subset of rows;   a division module to:
 determine a final subset of rows for each initial subset of rows, each final subset of rows comprising at least some of the one or more text rows of the corresponding initial subset of rows that have physical structures that are most similar to the most representative set of columns of a corresponding optimum set when compared to physical structures of all of the one or more text rows of the corresponding initial subset of rows; 
 determine a confidence factor for each final subset of rows, each confidence factor measuring a similarity of the physical structures of the at least some text rows in one corresponding final subset of rows to each other; and 
 determine a best confidence factor for each particular text row in the at least one document image, each particular text row having one or more confidence factors corresponding to one or more final subsets of rows in which the particular text row is an element; and 
   a classifier module to create one or more classes of text rows, each class comprising one or more particular text rows having a same best confidence factor.   
     
     
         4 . The system of  claim 3  wherein the optimum set module determines the optimum set for each initial subset of rows by:
 generating a histogram of column frequencies of the set of columns in the corresponding initial subset of rows, each column frequency comprising a number of times a particular column occurs in the corresponding initial subset of rows;   determining a threshold of the column frequencies for the corresponding initial subset of rows; and   selecting particular columns having the column frequency above the threshold to be included in the most representative set of columns for the corresponding optimum set.   
     
     
         5 . The system of  claim 3  wherein:
 each corresponding optimum set is represented by a corresponding master row having the set of columns from the corresponding initial subset of rows, each corresponding master row comprising a first indicator in particular columns that are elements of the corresponding optimum set and a second indicator in other particular columns in the set of columns for the corresponding initial subset of rows that are not elements of the corresponding optimum set; and   the division module:
 splits the one or more text rows in each corresponding initial subset of rows into at least a first group of text rows and a second group of text rows, the first group of text rows comprising the at least some of the one or more text rows, the at least some of the one or more text rows having at least smallest distances to the corresponding master row when compared to distances of all of the one or more text rows in the corresponding initial subset of rows, the second group of text rows comprising either no text rows or other text rows having at least larger distances to the corresponding master row when compared to the smallest distances of the at least some of the one or more text rows; and 
 selects the first group of text rows to be in the final subset of rows. 
   
     
     
         6 . The system of  claim 3  wherein:
 each corresponding optimum set is represented by a corresponding master row having the set of columns from the corresponding initial subset of rows, each corresponding master row comprising a first indicator in particular columns that are elements of the corresponding optimum set and a second indicator in other particular columns in the set of columns for the corresponding initial subset of rows that are not elements of the corresponding optimum set; and   the division module:
 splits the one or more text rows in each corresponding initial subset of rows into at least a first group of text rows and a second group of text rows, the first group of text rows comprising the at least some of the one or more text rows, the at least some of the one or more text rows having smallest distances and highest matches to the corresponding master row when compared to distances and matches of all of the one or more text rows in the corresponding initial subset of rows, the second group of text rows comprising either no text rows or other text rows having larger distances and smaller matches to the corresponding master row when compared to the smallest distances and highest matches of the at least some of the one or more text rows; and 
 selects the first group of text rows to be in the final subset of rows. 
   
     
     
         7 . The system of  claim 3  wherein:
 each optimum set is represented by a corresponding binary master row having the set of columns from the corresponding initial subset of rows, each corresponding binary master row comprising a binary 1 in particular columns that are elements of the corresponding optimum set and a binary 0 in other particular columns that are not elements of the corresponding optimum set; and   the division module comprises a thresholding module using a thresholding algorithm to:
 determine a distance threshold for each corresponding initial subset of rows for distances from each of the one or more text rows in each corresponding initial subset of rows to the corresponding master row; 
 split the one or more text rows in each corresponding initial subset of rows into at least a first group of text rows and a second group of text rows, the first group of text rows comprising first text rows with first distances under the threshold, the second group of text rows comprising either no text rows or second text rows with second distances not under the threshold; and 
 select the first group of text rows to be in the corresponding final subset of rows. 
   
     
     
         8 . The system of  claim 7  wherein:
 the thresholding module determines a final distances vector for each final subset of rows, each final distances vector comprising the first distances of the at least some of the one or more text rows of the corresponding final subset of rows to the corresponding master row; and   the confidence factor comprises a confidence factor ratio with at least one member of a another group consisting of:
 a rows frequency in the numerator and a variance of the first distances in the corresponding final distances vector in a denominator, the rows frequency comprising a number of the at least some of the one or more text rows in the corresponding final subset of rows; 
 the rows frequency and a master row length in the numerator and the variance and an average of the first distances in the corresponding final distances vector in the denominator; and 
 a quantity of a rows frequency cubed multiplied by the master row length in the numerator and another quantity of the variance multiplied by the average of the first distances in the final distances vector plus one in the denominator. 
   
     
     
         9 . The system of  claim 3  wherein:
 each optimum set is represented by a corresponding master row having the set of columns from the corresponding initial subset of rows, each corresponding master row comprising a binary vector with a binary 1 in particular columns that are elements of the corresponding optimum set and a binary 0 in other particular columns that are not elements of the corresponding optimum set; and   the division module comprises a clustering module using a clustering algorithm to:
 determine one or more clusters of one or more row points for each initial subset of rows, each row point corresponding to one of the one or more text rows in the corresponding initial subset of rows and comprising at least two members of a group consisting of a row distance, a row matches, and a row length for the one of the one or more text rows or normalized values thereof; 
 determine a cluster closeness value for each cluster in each initial subset of rows, each cluster closeness value based on a row matches average and a row distances average for text rows corresponding to row points of a corresponding cluster or normalized values thereof; 
 determine a final cluster for each initial subset of rows, each final cluster having a smallest cluster closeness value from cluster closeness values of the one or more clusters of the corresponding initial subset of rows, each final cluster comprising row points corresponding to at least some of the one or more text rows of the corresponding initial subset of rows; and 
 split the one or more text rows in each corresponding initial subset of rows into at least a final subset of rows and another subset of rows, the final subset of rows comprising the at least some of the one or more text rows of a corresponding final cluster for the corresponding initial subset of rows, the other subset of rows comprising either no text rows or one or more other text rows of the corresponding initial subset of rows that do not include the at least some of the one or more text rows. 
   
     
     
         10 . The system of  claim 9  wherein:
 the clustering module determines a final distances vector for each final subset of rows, each final distances vector comprising distances of the at least some of the one or more text rows of the corresponding final subset of rows to the corresponding master row associated with the corresponding final subset of rows, each final distances vector having a final row distances average; and   the clustering module determines a final matches vector for each final subset of rows, each final matches vector comprising, for each one of the at least some of the one or more text rows in the corresponding final subset of rows, a sum of matches between a binary 1 in the particular columns of the corresponding master row and another binary 1 in corresponding particular columns of the each one of the at least some of the one or more text rows, each final matches vector having a final row matches average; and   the confidence factor comprises a confidence factor ratio with at least one member of a group consisting of:
 a normalized rows frequency multiplied by the final row matches average of the final matches vector for the corresponding final subset of rows in a numerator and the final row distances average of the final distances vector for the corresponding final subset of rows in a denominator, the normalized rows frequency comprising a first number of the at least some of the one or more text rows in the corresponding final subset of rows divided by a second number of text rows in the document image; and 
 the final row matches average of the final matches vector in the numerator and the final row distances average of the final distances vector in the denominator. 
   
     
     
         11 . The system of  claim 10  wherein the clustering module determines two clusters of the one or more row points for each initial subset of rows. 
     
     
         12 . The system of  claim 3  wherein the division module comprises at least one member of a group consisting of:
 a thresholding module to determine the final subset of rows for each initial subset of rows using a thresholding algorithm; and   a clustering module to determine the final subset of rows for each initial subset of rows using a clustering algorithm.   
     
     
         13 . The system of  claim 3  wherein the modules further comprise a data extractor to extract data from at least one particular text row in at least one class. 
     
     
         14 . The system of  claim 13  wherein:
 each class has a class physical structure;   the system comprises memory comprising document model data for a plurality of document models and storing the at least one document image, the document model data identifying other class physical structures of other classes of the document models and regions of interest for the other classes of the document models; and   the data extractor:
 compares the class physical structures of the classes of the document image to the other class physical structures of the other classes for the document models to identify a matching document model; 
 when the matching document model is determined, determines a region of interest from the matching document model and extracts the data from a corresponding region of interest in the document image; and 
 when the matching document model is not determined, stores the class physical structures of the classes of the document image in memory as a new document model. 
   
     
     
         15 . The system of  claim 13  wherein the data extractor generates the extracted data to an output system or a user interface. 
     
     
         16 . The system of  claim 1  wherein the modules further comprise a data extractor to extract data from at least one text row in at least one class. 
     
     
         17 . The system of  claim 1  wherein the modules comprise:
 a line detector module to detect and remove lines when lines exist in the at least one document image and to save positions of vertical lines of the at least one document image in a vertical lines array when vertical lines exist in the at least one document image; and   an alignment system comprising a document block module to determine when at least one line pattern in the vertical lines array identifies at least two document blocks, to split the at least one document image into the at least two document blocks when the at least one line pattern is determined, and to vertically align the at least two document blocks before the classification system determines the columns.   
     
     
         18 . The system of  claim 1  wherein the modules further comprise:
 a line detector module to detect vertical lines; and   an alignment system comprising a line pattern module to:
 detect a center line splitting the at least one document image into at least two document blocks; 
 split the at least one document image into the at least two document blocks when the center line is detected; and 
 vertically align the at least two document blocks before the classification system determines the columns. 
   
     
     
         19 . The system of  claim 1  wherein:
 the modules further comprise an alignment system comprising a document block module to determine when at least one white space area is a white space divider that divides the at least one document image into at least two document blocks, to split the at least one document image into the at least two document blocks when the at least one white space is determined to be the white space divider, and to vertically align the at least two document blocks before the classification system determines the columns.   
     
     
         20 . The system of  claim 1  wherein the classification system comprises:
 a subsets module to determine the columns and to determine an initial subset of rows for each column having more than one character block aligned in that column in the text rows, each initial subset of rows comprising one or more text rows;   an optimum set module to determine an optimum set of columns for each initial subset of TOWS;   a division module to:
 determine a final subset of rows for each initial subset of rows, each final subset of rows comprising at least some of the one or more text rows of a corresponding initial subset of rows that have physical structures that are most similar to a corresponding optimum set when compared to physical structures of all of the one or more text rows in the corresponding initial subset of rows; 
 determine a confidence factor for each final subset of rows; and 
 determine a best confidence factor for each particular text row in the at least one document image; and 
   a classifier module to create one or more classes of text rows, each class comprising one or more particular text rows having a same best confidence factor.   
     
     
         21 . The system of  claim 20  wherein the division module comprises at least one member of a group consisting of:
 a thresholding module to determine the final subset of rows for each initial subset of rows using a thresholding algorithm; and   a clustering module to determine the final subset of rows for each initial subset of rows using a clustering algorithm.   
     
     
         22 . A system to process at least one document image comprising a plurality of text rows and a plurality of characters, each text row having at least one character, the system comprising:
 at least one processor; and   a plurality of modules to execute on the at least one processor, the modules comprising:
 a character block creator to create character blocks for the characters in the text rows and to determine spatial positions of alignments for the character blocks, each text row having a physical structure defined by one or more spatial positions of one or more alignments of one or more character blocks in that text row; and 
 a classification system to assign the text rows to classes based on a similarity of the physical structure of each text row as defined by the one or more spatial positions of the one or more alignments of the one or more character blocks therein. 
   
     
     
         23 . The system of  claim 22  wherein the classification system comprises:
 a subsets module to determine the spatial positions for the alignments of the character blocks and to determine an initial subset of rows for each alignment spatial position having more than one character block aligned in that alignment spatial position in the text rows, each initial subset of rows comprising one or more text rows having at least one alignment of at least one character block in a selected alignment spatial position, each initial subset of rows having a set of alignment spatial positions comprising the selected alignment spatial position and other alignment spatial positions in the one or more text rows therein;   an optimum set module to determine an optimum set for each initial subset of rows, each optimum set comprising a most representative set of alignment spatial positions selected from the set of alignment spatial positions of a corresponding initial subset of rows;   a division module to:
 determine a final subset of rows for each initial subset of rows, each final subset of rows comprising at least some of the one or more text rows of the corresponding initial subset of rows that have physical structures that are most similar to the most representative set of alignment spatial positions of a corresponding optimum set when compared to physical structures of all of the one or more text rows of the corresponding initial subset of rows; 
 determine a confidence factor for each final subset of rows, each confidence factor measuring a similarity of the physical structures of the at least some text rows in one corresponding final subset of rows to each other; and 
 determine a best confidence factor for each particular text row in the at least one document image, each particular text row having one or more confidence factors corresponding to one or more final subsets of rows in which the particular text row is an element; and 
   a classifier module to create one or more classes of text rows, each class comprising one or more particular text rows having a same best confidence factor.   
     
     
         24 . The system of  claim 22  wherein the optimum set module determines the optimum set for each initial subset of rows by:
 generating a histogram of alignment spatial position frequencies of the set of alignment spatial positions in the corresponding initial subset of rows, each alignment spatial position frequency comprising a number of times a particular alignment spatial position occurs in the corresponding initial subset of rows;   determining a threshold of the alignment spatial position frequencies for the corresponding initial subset of rows; and   selecting particular alignment spatial positions having the alignment spatial position frequency above the threshold to be included in the most representative set of alignment spatial positions for the corresponding optimum set.   
     
     
         25 . The system of  claim 22  wherein:
 each optimum set is represented by a corresponding binary master row having the set of alignment spatial positions from the corresponding initial subset of rows, each corresponding binary master row comprising a binary 1 in particular alignment spatial positions that are elements of the corresponding optimum set and a binary 0 in other particular alignment spatial positions that are not elements of the corresponding optimum set; and   the division module comprises a thresholding module using a thresholding algorithm to:
 determine a distance threshold for each corresponding initial subset of rows for distances from each of the one or more text rows in each corresponding initial subset of rows to the corresponding master row; 
 split the one or more text rows in each corresponding initial subset of rows into at least a first group of text rows and a second group of text rows, the first group of text rows comprising first text rows with first distances under the threshold, the second group of text rows comprising either no text rows or second text rows with second distances not under the threshold; and 
 select the first group of text rows to be in the corresponding final subset of rows. 
   
     
     
         26 . The system of  claim 25  wherein:
 the thresholding module determines a final distances vector for each final subset of rows, each final distances vector comprising the first distances of the at least some of the one or more text rows of the corresponding final subset of rows to the corresponding master row; and   the confidence factor comprises a confidence factor ratio with at least one member of a another group consisting of:
 a rows frequency in the numerator and a variance of the first distances in the corresponding final distances vector in a denominator, the rows frequency comprising a number of the at least some of the one or more text rows in the corresponding final subset of rows; 
 the rows frequency and a master row length in the numerator and the variance and an average of the first distances in the corresponding final distances vector in the denominator; and 
 a quantity of a rows frequency cubed multiplied by the master row length in the numerator and another quantity of the variance multiplied by the average of the first distances in the final distances vector plus one in the denominator. 
   
     
     
         27 . The system of  claim 22  wherein:
 each optimum set is represented by a corresponding master row having the set of alignment spatial positions from the corresponding initial subset of rows, each corresponding master row comprising a binary vector with a binary 1 in particular alignment spatial positions that are elements of the corresponding optimum set and a binary 0 in other particular alignment spatial positions that are not elements of the corresponding optimum set; and   the division module comprises a clustering module using a clustering algorithm to:
 determine one or more clusters of one or more row points for each initial subset of rows, each row point corresponding to one of the one or more text rows in the corresponding initial subset of rows and comprising at least two members of a group consisting of a row distance, a row matches, and a row length for the one of the one or more text rows or normalized values thereof; 
 determine a cluster closeness value for each cluster in each initial subset of rows, each cluster closeness value based on a row matches average and a row distances average for text rows corresponding to row points of a corresponding cluster or normalized values thereof; 
 determine a final cluster for each initial subset of rows, each final cluster having a smallest cluster closeness value from cluster closeness values of the one or more clusters of the corresponding initial subset of rows, each final cluster comprising row points corresponding to at least some of the one or more text rows of the corresponding initial subset of rows; and 
 split the one or more text rows in each corresponding initial subset of rows into at least a final subset of rows and another subset of rows, the final subset of rows comprising the at least some of the one or more text rows of a corresponding final cluster for the corresponding initial subset of rows, the other subset of rows comprising either no text rows or one or more other text rows of the corresponding initial subset of rows that do not include the at least some of the one or more text rows. 
   
     
     
         28 . The system of  claim 27  wherein:
 the clustering module determines a final distances vector for each final subset of rows, each final distances vector comprising distances of the at least some of the one or more text rows of the corresponding final subset of rows to the corresponding master row associated with the corresponding final subset of rows, each final distances vector having a final row distances average; and   the clustering module determines a final matches vector for each final subset of rows, each final matches vector comprising, for each one of the at least some of the one or more text rows in the corresponding final subset of rows, a sum of matches between a binary 1 in the particular alignment spatial positions of the corresponding master row and another binary 1 in corresponding particular alignment spatial positions of the each one of the at least some of the one or more text rows, each final matches vector having a final row matches average; and   the confidence factor comprises a confidence factor ratio with at least one member of a group consisting of:
 a normalized rows frequency multiplied by the final row matches average of the final matches vector for the corresponding final subset of rows in a numerator and the final row distances average of the final distances vector for the corresponding final subset of rows in a denominator, the normalized rows frequency comprising a first number of the at least some of the one or more text rows in the corresponding final subset of rows divided by a second number of text rows in the document image; and 
 the final row matches average of the final matches vector in the numerator and the final row distances average of the final distances vector in the denominator. 
   
     
     
         29 . The system of  claim 22  wherein the modules further comprise a data extractor to extract data from at least one particular text row in at least one class. 
     
     
         30 . The system of  claim 29  wherein the data extractor generates the extracted data to an output system or a user interface. 
     
     
         31 . The system of  claim 22  wherein the classification system comprises:
 a subsets module to determine the alignment spatial positions and to determine an initial subset of rows for each alignment spatial position having more than one character block aligned in that alignment spatial position in the text rows, each initial subset of rows comprising one or more text rows;   an optimum set module to determine an optimum set of alignment spatial positions for each initial subset of rows;   a division module to:
 determine a final subset of rows for each initial subset of rows, each final subset of rows comprising at least some of the one or more text rows of a corresponding initial subset of rows that have physical structures that are most similar to a corresponding optimum set when compared to physical structures of all of the one or more text rows in the corresponding initial subset of rows; 
 determine a confidence factor for each final subset of rows; and 
 determine a best confidence factor for each particular text row in the at least one document image; and 
   a classifier module to create one or more classes of text rows, each class comprising one or more particular text rows having a same best confidence factor.   
     
     
         32 . The system of  claim 31  wherein the division module comprises at least one member of a group consisting of:
 a thresholding module to determine the final subset of rows for each initial subset of rows using a thresholding algorithm; and   a clustering module to determine the final subset of rows for each initial subset of rows using a clustering algorithm.   
     
     
         33 . A system to process at least one document image comprising a plurality of text rows and a plurality of characters, each text row having at least one character, the system comprising:
 at least one processor; and   a plurality of modules to execute on the at least one processor, the modules comprising:
 a character block creator to determine character blocks for the characters in the text rows and to determine spatial positions of alignments of the character blocks, the spatial positions of the alignments of the character blocks in each text row defining a physical structure of that text row; and 
 a classification system to determine classes for the text rows based on the physical structure of each text row as defined by the spatial positions of the alignments of the character blocks in that text row. 
   
     
     
         34 . A processing system comprising:
 memory comprising at least one document image comprising a plurality of text rows and a plurality of characters, each text row having at least one character;   at least one processor; and   a plurality of modules to execute on the at least one processor, the modules comprising:
 a first module to create character blocks for the characters in the text rows; 
 a second module to determine positions of alignments for the character blocks, each text row having a physical structure defined by one or more positions of one or more alignments of one or more character blocks in that text row; and 
 a third module to assign the text rows to classes and to assign particular text rows having a most similar physical structure to a same class when compared to the physical structures of all text rows of the document. 
   
     
     
         35 . A computer-readable medium encoded with a system to process at least one document image comprising a plurality of text rows and a plurality of characters, each text row having at least one character, the system comprising a plurality of modules to execute on at least one processor, the modules comprising:
 a character block creator to create character blocks for the characters in the text rows and to determine positions of alignments of the character blocks; and   a classification system to determine columns for the alignments of the character blocks at the positions of the alignments, each text row having a physical structure defined by the columns of the alignments of the character blocks in that text row, and to determine classes for the text rows based on the physical structures of the text rows as defined by the columns of the character blocks in each text row.   
     
     
         36 . The system of  claim 35  wherein the classification system determines the classes by:
 determining an initial subset of rows for each column having more than one character block aligned in that column in the text rows, each initial subset of rows comprising one or more text rows having at least one alignment of at least one character block in a selected column, each initial subset of rows having a set of columns comprising the selected column and other columns in the one or more text rows therein;   determining an optimum set for each initial subset of rows, each optimum set comprising a most representative set of columns selected from the set of columns of one of the initial subsets of rows;   determining a final subset of rows for each initial subset of rows, each final subset of rows comprising at least some of the one or more text rows of a corresponding initial subset of rows that have physical structures that are most similar to a corresponding most representative set of columns of a corresponding optimum set when compared to physical structures of all of the one or more text rows of the corresponding initial subset of rows;   determining a confidence factor for each final subset of rows, each confidence factor measuring a similarity of the physical structures of each one of the at least some text rows in one corresponding final subset of rows to each other one of the at least some text rows in the one corresponding final subset of rows;   determining a best confidence factor for each particular text row in the at least one document image, each particular text row having one or more confidence factors corresponding to one or more final subsets of rows in which the particular text row is an element; and   creating one or more classes of text rows, each class comprising one or more particular text rows having a same best confidence factor.   
     
     
         37 . The system of  claim 35  wherein the classification system comprises:
 a subsets module to determine the columns for the alignments of the character blocks and to determine an initial subset of rows for each column having more than one character block aligned in that column in the text rows, each initial subset of rows comprising one or more text rows having at least one alignment of at least one character block in a selected column, each initial subset of rows having a set of columns comprising the selected column and other columns in the one or more text rows therein;   an optimum set module to determine an optimum set for each initial subset of rows, each optimum set comprising a most representative set of columns selected from the set of columns of a corresponding initial subset of rows;   a division module to:
 determine a final subset of rows for each initial subset of rows, each final subset of rows comprising at least some of the one or more text rows of the corresponding initial subset of rows that have physical structures that are most similar to the most representative set of columns of a corresponding optimum set when compared to physical structures of all of the one or more text rows of the corresponding initial subset of rows; 
 determine a confidence factor for each final subset of rows, each confidence factor measuring a similarity of the physical structures of the at least some text rows in one corresponding final subset of rows to each other; and 
 determine a best confidence factor for each particular text row in the at least one document image, each particular text row having one or more confidence factors corresponding to one or more final subsets of rows in which the particular text row is an element; and 
   a classifier module to create one or more classes of text rows, each class comprising one or more particular text rows having a same best confidence factor.   
     
     
         38 . The system of  claim 37  wherein the optimum set module determines the optimum set for each initial subset of rows by:
 generating a histogram of column frequencies of the set of columns in the corresponding initial subset of rows, each column frequency comprising a number of times a particular column occurs in the corresponding initial subset of rows;   determining a threshold of the column frequencies for the corresponding initial subset of rows; and   selecting particular columns having the column frequency above the threshold to be included in the most representative set of columns for the corresponding optimum set.   
     
     
         39 . The system of  claim 37  wherein:
 each corresponding optimum set is represented by a corresponding master row having the set of columns from the corresponding initial subset of rows, each corresponding master row comprising a first indicator in particular columns that are elements of the corresponding optimum set and a second indicator in other particular columns in the set of columns for the corresponding initial subset of rows that are not elements of the corresponding optimum set; and   the division module:
 splits the one or more text rows in each corresponding initial subset of rows into at least a first group of text rows and a second group of text rows, the first group of text rows comprising the at least some of the one or more text rows, the at least some of the one or more text rows having at least smallest distances to the corresponding master row when compared to distances of all of the one or more text rows in the corresponding initial subset of rows, the second group of text rows comprising either no text rows or other text rows having at least larger distances to the corresponding master row when compared to the smallest distances of the at least some of the one or more text rows; and 
   selects the first group of text rows to be in the final subset of rows.   
     
     
         40 . The system of  claim 37  wherein:
 each corresponding optimum set is represented by a corresponding master row having the set of columns from the corresponding initial subset of rows, each corresponding master row comprising a first indicator in particular columns that are elements of the corresponding optimum set and a second indicator in other particular columns in the set of columns for the corresponding initial subset of rows that are not elements of the corresponding optimum set; and   the division module:
 splits the one or more text rows in each corresponding initial subset of rows into at least a first group of text rows and a second group of text rows, the first group of text rows comprising the at least some of the one or more text rows, the at least some of the one or more text rows having smallest distances and highest matches to the corresponding master row when compared to distances and matches of all of the one or more text rows in the corresponding initial subset of rows, the second group of text rows comprising either no text rows or other text rows having larger distances and smaller matches to the corresponding master row when compared to the smallest distances and highest matches of the at least some of the one or more text rows; and 
 selects the first group of text rows to be in the final subset of rows. 
   
     
     
         41 . The system of  claim 37  wherein:
 each optimum set is represented by a corresponding binary master row having the set of columns from the corresponding initial subset of rows, each corresponding binary master row comprising a binary 1 in particular columns that are elements of the corresponding optimum set and a binary 0 in other particular columns that are not elements of the corresponding optimum set; and   the division module comprises a thresholding module using a thresholding algorithm to:
 determine a distance threshold for each corresponding initial subset of rows for distances from each of the one or more text rows in each corresponding initial subset of rows to the corresponding master row; 
 split the one or more text rows in each corresponding initial subset of rows into at least a first group of text rows and a second group of text rows, the first group of text rows comprising first text rows with first distances under the threshold, the second group of text rows comprising either no text rows or second text rows with second distances not under the threshold; and 
 select the first group of text rows to be in the corresponding final subset of rows. 
   
     
     
         42 . The system of  claim 41  wherein:
 the thresholding module determines a final distances vector for each final subset of rows, each final distances vector comprising the first distances of the at least some of the one or more text rows of the corresponding final subset of rows to the corresponding master row; and   the confidence factor comprises a confidence factor ratio with at least one member of a another group consisting of:
 a rows frequency in the numerator and a variance of the first distances in the corresponding final distances vector in a denominator, the rows frequency comprising a number of the at least some of the one or more text rows in the corresponding final subset of rows; 
 the rows frequency and a master row length in the numerator and the variance and an average of the first distances in the corresponding final distances vector in the denominator; and 
 a quantity of a rows frequency cubed multiplied by the master row length in the numerator and another quantity of the variance multiplied by the average of the first distances in the final distances vector plus one in the denominator. 
   
     
     
         43 . The system of  claim 37  wherein:
 each optimum set is represented by a corresponding master row having the set of columns from the corresponding initial subset of rows, each corresponding master row comprising a binary vector with a binary 1 in particular columns that are elements of the corresponding optimum set and a binary 0 in other particular columns that are not elements of the corresponding optimum set; and   the division module comprises a clustering module using a clustering algorithm to:
 determine one or more clusters of one or more row points for each initial subset of rows, each row point corresponding to one of the one or more text rows in the corresponding initial subset of rows and comprising at least two members of a group consisting of a row distance, a row matches, and a row length for the one of the one or more text rows or normalized values thereof; 
 determine a cluster closeness value for each cluster in each initial subset of rows, each cluster closeness value based on a row matches average and a row distances average for text rows corresponding to row points of a corresponding cluster or normalized values thereof; 
 determine a final cluster for each initial subset of rows, each final cluster having a smallest cluster closeness value from cluster closeness values of the one or more clusters of the corresponding initial subset of rows, each final cluster comprising row points corresponding to at least some of the one or more text rows of the corresponding initial subset of rows; and 
 split the one or more text rows in each corresponding initial subset of rows into at least a final subset of rows and another subset of rows, the final subset of rows comprising the at least some of the one or more text rows of a corresponding final cluster for the corresponding initial subset of rows, the other subset of rows comprising either no text rows or one or more other text rows of the corresponding initial subset of rows that do not include the at least some of the one or more text rows. 
   
     
     
         44 . The system of  claim 43  wherein:
 the clustering module determines a final distances vector for each final subset of rows, each final distances vector comprising distances of the at least some of the one or more text rows of the corresponding final subset of rows to the corresponding master row associated with the corresponding final subset of rows, each final distances vector having a final row distances average; and   the clustering module determines a final matches vector for each final subset of rows, each final matches vector comprising, for each one of the at least some of the one or more text rows in the corresponding final subset of rows, a sum of matches between a binary 1 in the particular columns of the corresponding master row and another binary 1 in corresponding particular columns of the each one of the at least some of the one or more text rows, each final matches vector having a final row matches average; and   the confidence factor comprises a confidence factor ratio with at least one member of a group consisting of:
 a normalized rows frequency multiplied by the final row matches average of the final matches vector for the corresponding final subset of rows in a numerator and the final row distances average of the final distances vector for the corresponding final subset of rows in a denominator, the normalized rows frequency comprising a first number of the at least some of the one or more text rows in the corresponding final subset of rows divided by a second number of text rows in the document image; and 
 the final row matches average of the final matches vector in the numerator and the final row distances average of the final distances vector in the denominator. 
   
     
     
         45 . The system of  claim 44  wherein the clustering module determines two clusters of the one or more row points for each initial subset of rows. 
     
     
         46 . The system of  claim 37  wherein the division module comprises at least one member of a group consisting of:
 a thresholding module to determine the final subset of rows for each initial subset of rows using a thresholding algorithm; and   a clustering module to determine the final subset of rows for each initial subset of rows using a clustering algorithm.   
     
     
         47 . The system of  claim 37  wherein the modules further comprise a data extractor to extract data from at least one particular text row in at least one class. 
     
     
         48 . The system of  claim 47  wherein:
 each class has a class physical structure;   the system comprises memory comprising document model data for a plurality of document models and storing the at least one document image, the document model data identifying other class physical structures of other classes of the document models and regions of interest for the other classes of the document models; and   the data extractor:
 compares the class physical structures of the classes of the document image to the other class physical structures of the other classes for the document models to identify a matching document model; 
 when the matching document model is determined, determines a region of interest from the matching document model and extracts the data from a corresponding region of interest in the document image; and 
 when the matching document model is not determined, stores the class physical structures of the classes of the document image in memory as a new document model. 
   
     
     
         49 . The system of  claim 47  wherein the data extractor generates the extracted data to an output system or a user interface. 
     
     
         50 . The system of  claim 35  wherein the modules further comprise a data extractor to extract data from at least one text row in at least one class. 
     
     
         51 . The system of  claim 35  wherein the modules comprise:
 a line detector module to detect and remove lines when lines exist in the at least one document image and to save positions of vertical lines of the at least one document image in a vertical lines array when vertical lines exist in the at least one document image; and   an alignment system comprising a document block module to determine when at least one line pattern in the vertical lines array identifies at least two document blocks, to split the at least one document image into the at least two document blocks when the at least one line pattern is determined, and to vertically align the at least two document blocks before the classification system determines the columns.   
     
     
         52 . The system of  claim 35  wherein the modules further comprise:
 a line detector module to detect vertical lines; and   an alignment system comprising a line pattern module to:
 detect a center line splitting the at least one document image into at least two document blocks; 
 split the at least one document image into the at least two document blocks when the center line is detected; and 
 vertically align the at least two document blocks before the classification system determines the columns. 
   
     
     
         53 . The system of  claim 35  wherein:
 the modules further comprise an alignment system comprising a document block module to determine when at least one white space area is a white space divider that divides the at least one document image into at least two document blocks, to split the at least one document image into the at least two document blocks when the at least one white space is determined to be the white space divider, and to vertically align the at least two document blocks before the classification system determines the columns.   
     
     
         54 . The system of  claim 35  wherein the classification system comprises:
 a subsets module to determine the columns and to determine an initial subset of rows for each column having more than one character block aligned in that column in the text rows, each initial subset of rows comprising one or more text rows;   an optimum set module to determine an optimum set of columns for each initial subset of TOWS;   a division module to:
 determine a final subset of rows for each initial subset of rows, each final subset of rows comprising at least some of the one or more text rows of a corresponding initial subset of rows that have physical structures that are most similar to a corresponding optimum set when compared to physical structures of all of the one or more text rows in the corresponding initial subset of rows; 
 determine a confidence factor for each final subset of rows; and 
 determine a best confidence factor for each particular text row in the at least one document image; and 
   a classifier module to create one or more classes of text rows, each class comprising one or more particular text rows having a same best confidence factor.   
     
     
         55 . The system of  claim 54  wherein the division module comprises at least one member of a group consisting of:
 a thresholding module to determine the final subset of rows for each initial subset of rows using a thresholding algorithm; and   a clustering module to determine the final subset of rows for each initial subset of rows using a clustering algorithm.   
     
     
         56 . A computer-readable medium encoded with a system to process at least one document image comprising a plurality of text rows and a plurality of characters, each text row having at least one character, the system comprising a plurality of modules to execute on at least one processor, the modules comprising:
 a character block creator to create character blocks for the characters in the text rows and to determine spatial positions of alignments for the character blocks, each text row having a physical structure defined by one or more spatial positions of one or more alignments of one or more character blocks in that text row; and   a classification system to assign the text rows to classes based on a similarity of the physical structure of each text row as defined by the one or more spatial positions of the one or more alignments of the one or more character blocks therein.   
     
     
         57 . The system of  claim 56  wherein the classification system comprises:
 a subsets module to determine the spatial positions for the alignments of the character blocks and to determine an initial subset of rows for each alignment spatial position having more than one character block aligned in that alignment spatial position in the text rows, each initial subset of rows comprising one or more text rows having at least one alignment of at least one character block in a selected alignment spatial position, each initial subset of rows having a set of alignment spatial positions comprising the selected alignment spatial position and other alignment spatial positions in the one or more text rows therein;   an optimum set module to determine an optimum set for each initial subset of rows, each optimum set comprising a most representative set of alignment spatial positions selected from the set of alignment spatial positions of a corresponding initial subset of rows;   a division module to:
 determine a final subset of rows for each initial subset of rows, each final subset of rows comprising at least some of the one or more text rows of the corresponding initial subset of rows that have physical structures that are most similar to the most representative set of alignment spatial positions of a corresponding optimum set when compared to physical structures of all of the one or more text rows of the corresponding initial subset of rows; 
 determine a confidence factor for each final subset of rows, each confidence factor measuring a similarity of the physical structures of the at least some text rows in one corresponding final subset of rows to each other; and 
 determine a best confidence factor for each particular text row in the at least one document image, each particular text row having one or more confidence factors corresponding to one or more final subsets of rows in which the particular text row is an element; and 
   a classifier module to create one or more classes of text rows, each class comprising one or more particular text rows having a same best confidence factor.   
     
     
         58 . The system of  claim 56  wherein the optimum set module determines the optimum set for each initial subset of rows by:
 generating a histogram of alignment spatial position frequencies of the set of alignment spatial positions in the corresponding initial subset of rows, each alignment spatial position frequency comprising a number of times a particular alignment spatial position occurs in the corresponding initial subset of rows;   determining a threshold of the alignment spatial position frequencies for the corresponding initial subset of rows; and   selecting particular alignment spatial positions having the alignment spatial position frequency above the threshold to be included in the most representative set of alignment spatial positions for the corresponding optimum set.   
     
     
         59 . The system of  claim 56  wherein:
 each optimum set is represented by a corresponding binary master row having the set of alignment spatial positions from the corresponding initial subset of rows, each corresponding binary master row comprising a binary 1 in particular alignment spatial positions that are elements of the corresponding optimum set and a binary 0 in other particular alignment spatial positions that are not elements of the corresponding optimum set; and   the division module comprises a thresholding module using a thresholding algorithm to:
 determine a distance threshold for each corresponding initial subset of rows for distances from each of the one or more text rows in each corresponding initial subset of rows to the corresponding master row; 
 split the one or more text rows in each corresponding initial subset of rows into at least a first group of text rows and a second group of text rows, the first group of text rows comprising first text rows with first distances under the threshold, the second group of text rows comprising either no text rows or second text rows with second distances not under the threshold; and 
   select the first group of text rows to be in the corresponding final subset of rows.   
     
     
         60 . The system of  claim 59  wherein:
 the thresholding module determines a final distances vector for each final subset of rows, each final distances vector comprising the first distances of the at least some of the one or more text rows of the corresponding final subset of rows to the corresponding master row; and   the confidence factor comprises a confidence factor ratio with at least one member of a another group consisting of:
 a rows frequency in the numerator and a variance of the first distances in the corresponding final distances vector in a denominator, the rows frequency comprising a number of the at least some of the one or more text rows in the corresponding final subset of rows; 
 the rows frequency and a master row length in the numerator and the variance and an average of the first distances in the corresponding final distances vector in the denominator; and 
 a quantity of a rows frequency cubed multiplied by the master row length in the numerator and another quantity of the variance multiplied by the average of the first distances in the final distances vector plus one in the denominator. 
   
     
     
         61 . The system of  claim 56  wherein:
 each optimum set is represented by a corresponding master row having the set of alignment spatial positions from the corresponding initial subset of rows, each corresponding master row comprising a binary vector with a binary 1 in particular alignment spatial positions that are elements of the corresponding optimum set and a binary 0 in other particular alignment spatial positions that are not elements of the corresponding optimum set; and   the division module comprises a clustering module using a clustering algorithm to:
 determine one or more clusters of one or more row points for each initial subset of rows, each row point corresponding to one of the one or more text rows in the corresponding initial subset of rows and comprising at least two members of a group consisting of a row distance, a row matches, and a row length for the one of the one or more text rows or normalized values thereof; 
 determine a cluster closeness value for each cluster in each initial subset of rows, each cluster closeness value based on a row matches average and a row distances average for text rows corresponding to row points of a corresponding cluster or normalized values thereof; 
 determine a final cluster for each initial subset of rows, each final cluster having a smallest cluster closeness value from cluster closeness values of the one or more clusters of the corresponding initial subset of rows, each final cluster comprising row points corresponding to at least some of the one or more text rows of the corresponding initial subset of rows; and 
 split the one or more text rows in each corresponding initial subset of rows into at least a final subset of rows and another subset of rows, the final subset of rows comprising the at least some of the one or more text rows of a corresponding final cluster for the corresponding initial subset of rows, the other subset of rows comprising either no text rows or one or more other text rows of the corresponding initial subset of rows that do not include the at least some of the one or more text rows. 
   
     
     
         62 . The system of  claim 61  wherein:
 the clustering module determines a final distances vector for each final subset of rows, each final distances vector comprising distances of the at least some of the one or more text rows of the corresponding final subset of rows to the corresponding master row associated with the corresponding final subset of rows, each final distances vector having a final row distances average; and   the clustering module determines a final matches vector for each final subset of rows, each final matches vector comprising, for each one of the at least some of the one or more text rows in the corresponding final subset of rows, a sum of matches between a binary 1 in the particular alignment spatial positions of the corresponding master row and another binary 1 in corresponding particular alignment spatial positions of the each one of the at least some of the one or more text rows, each final matches vector having a final row matches average; and   the confidence factor comprises a confidence factor ratio with at least one member of a group consisting of:
 a normalized rows frequency multiplied by the final row matches average of the final matches vector for the corresponding final subset of rows in a numerator and the final row distances average of the final distances vector for the corresponding final subset of rows in a denominator, the normalized rows frequency comprising a first number of the at least some of the one or more text rows in the corresponding final subset of rows divided by a second number of text rows in the document image; and 
 the final row matches average of the final matches vector in the numerator and the final row distances average of the final distances vector in the denominator. 
   
     
     
         63 . The system of  claim 56  wherein the modules further comprise a data extractor to extract data from at least one particular text row in at least one class. 
     
     
         64 . The system of  claim 63  wherein the data extractor generates the extracted data to an output system or a user interface. 
     
     
         65 . The system of  claim 56  wherein the classification system comprises:
 a subsets module to determine the alignment spatial positions and to determine an initial subset of rows for each alignment spatial position having more than one character block aligned in that alignment spatial position in the text rows, each initial subset of rows comprising one or more text rows;   an optimum set module to determine an optimum set of alignment spatial positions for each initial subset of rows;   a division module to:
 determine a final subset of rows for each initial subset of rows, each final subset of rows comprising at least some of the one or more text rows of a corresponding initial subset of rows that have physical structures that are most similar to a corresponding optimum set when compared to physical structures of all of the one or more text rows in the corresponding initial subset of rows; 
 determine a confidence factor for each final subset of rows; and 
 determine a best confidence factor for each particular text row in the at least one document image; and 
   a classifier module to create one or more classes of text rows, each class comprising one or more particular text rows having a same best confidence factor.   
     
     
         66 . The system of  claim 65  wherein the division module comprises at least one member of a group consisting of:
 a thresholding module to determine the final subset of rows for each initial subset of rows using a thresholding algorithm; and   a clustering module to determine the final subset of rows for each initial subset of rows using a clustering algorithm.   
     
     
         67 . A computer-readable medium encoded with a system to process at least one document image comprising a plurality of text rows and a plurality of characters, each text row having at least one character, the system comprising a plurality of modules to execute on at least one processor, the modules comprising:
 a character block creator to determine character blocks for the characters in the text rows and to determine spatial positions of alignments of the character blocks, the spatial positions of the alignments of the character blocks in each text row defining a physical structure of that text row; and   a classification system to determine classes for the text rows based on the physical structure of each text row as defined by the spatial positions of the alignments of the character blocks in that text row.   
     
     
         68 . A computer-readable medium encoded with a system to process at least one document image comprising a plurality of text rows and a plurality of characters, each text row having at least one character, the system comprising a plurality of modules to execute on at least one processor, the modules comprising:
 a first module to create character blocks for the characters in the text rows;   a second module to determine positions of alignments for the character blocks, each text row having a physical structure defined by one or more positions of one or more alignments of one or more character blocks in that text row; and   a third module to assign the text rows to classes and to assign particular text rows having a most similar physical structure to a same class when compared to the physical structures of all text rows of the document.   
     
     
         69 . A computer-readable medium encoded with instructions that when executed by at least one processor process at least one document image comprising a plurality of text rows and a plurality of characters, each text row having at least one character, the instructions performing operations comprising:
 create character blocks for the characters in the text rows and to determine positions of alignments of the character blocks;   determine columns for the alignments of the character blocks at the positions of the alignments, each text row having a physical structure defined by the columns of the alignments of the character blocks in that text row; and   determine classes for the text rows based on the physical structures of the text rows as defined by the columns of the character blocks in each text row.   
     
     
         70 . A computer-readable medium encoded with a system to process at least one document image comprising a plurality of text rows and a plurality of characters, each text row having at least one character, the system comprising instructions that when executed by at least one processor perform operations comprising:
 create character blocks for the characters in the text rows;   determine spatial positions of alignments for the character blocks, each text row having a physical structure defined by one or more spatial positions of one or more alignments of one or more character blocks in that text row; and   assign the text rows to classes based on a similarity of the physical structure of each text row as defined by the one or more spatial positions of the one or more alignments of the one or more character blocks therein.   
     
     
         71 . A computer-readable medium encoded with instructions that when executed by at least one processor process at least one document image comprising a plurality of text rows and a plurality of characters, each text row having at least one character, the instructions performing operations comprising:
 determine character blocks for the characters in the text rows;   determine spatial positions of alignments of the character blocks, the spatial positions of the alignments of the character blocks in each text row defining a physical structure of that text row; and   determine classes for the text rows based on the physical structure of each text row as defined by the spatial positions of the alignments of the character blocks in that text row.   
     
     
         72 . A machine-readable medium encoded with instructions that when executed by at least one processor process at least one document image comprising a plurality of text rows and a plurality of characters, each text row having at least one character, the instructions performing operations comprising:
 create character blocks for the characters in the text rows;   determine positions of alignments for the character blocks, each text row having a physical structure defined by one or more positions of one or more alignments of one or more character blocks in that text row; and   assign the text rows to classes and to assign particular text rows having a most similar physical structure to a same class when compared to the physical structures of all text rows of the document.