Interactive visualization of big data sets and models including textual data

Abstract

Systems and processes are disclosed for advanced text analysis in the field of big data analytics and visualization: Users can now factor text into their predictive models, alongside regression, time/date and categorical information. This is ideal for building models where text content may play a prominent role (e.g., social media or customer service logs). Multiple data types, including text fields, may be combined together in datasets and models, and may be presented in various interactive visualization displays.

1 . A method comprising:
 accessing a set of sample data instances, each instance comprising a corresponding value for each one of a plurality of data fields;   processing the sample data instances so as to form a dataset; and   generating a visual summary of the dataset on a computing device, the visual representation comprising a generally tabular presentation, the presentation including a series of rows and columns of information, each row corresponding to one of the data fields of the sample data, and each column displaying a corresponding parameter in each of the rows;   wherein at least one of the columns displays, in each row of the presentation, an indication to identify a data type of the corresponding row; and wherein   the single data type is detected from a set of data types that includes at least a text data type.   
     
     
         2 . The method of  claim 1  wherein processing the sample data includes tokenizing the text fields. 
     
     
         3 . The method of  claim 1  wherein:
 at least one of the columns displays, in each row, an indication of a corresponding number of instances of the sample data that include the corresponding data field; 
 at least one of the columns displays, in each row, a corresponding number of instances of the sample data that are missing the corresponding data field; and 
 at least one of the columns displays, in each row, a corresponding number of instances of the sample data that have an error in the corresponding data field. 
 
     
     
         4 . The method of  claim 1 :
 wherein at least one of the columns displays, in each row, a corresponding histogram of the values of the corresponding data field in the instances of the sample data.   
     
     
         5 . The method of  claim 3  and further comprising:
 responsive to a user input, generating a tag cloud for a selected text field, and displaying the tag cloud on an electronic display, wherein the tag cloud displays a plurality of the words [tokens] of the corresponding text field; and 
 wherein the words are displayed in the tag cloud in font sizes that are selected in proportion to the respective frequency of occurrence of each word in the corresponding text field. 
 
     
     
         6 . The method of  claim 5  and further comprising:
 responsive to user selection of a word in the tag cloud display, further displaying a number of occurrences of the selected word in a popup overlying or adjacent to the tag cloud display. 
 
     
     
         7 . A method comprising:
 accessing a digital source data file comprising a plurality of rows or records, each record comprising at least one data field;   processing the source data file on a computing device to recognize a data type for each of the data fields;   in the case of recognizing a text data type, matching the text to a human language;   for each one of the recognized text fields in the source data file, applying a stemming process corresponding to the matched human language, thereby tokenizing the text data fields;   utilizing the tokenized data fields in forming a dataset based on the source file; and   displaying an interactive summary of the dataset on a display screen of the computing device.   
     
     
         8 . The method of  claim 7 , further comprising:
 building a model based at least in part on the dataset;   accessing a model based at least in part on a dataset comprising data items;   receiving an indication of a type of visualization to be displayed;   generating a space-filling graphical representation of the model on a computing device, the space-filling graphical representation comprising a plurality of segments arranged to realize the indicated type of visualization; and   displaying the space-filling graphical representation of the model on a display screen of the computing device.   
     
     
         9 . The method of  claim 8 , further comprising:
 providing a color scheme based at least in part on the indicated type of visualization; and   displaying the graphical representation of the model on the display screen using the color scheme.   
     
     
         10 . The method of  claim 7 , wherein the interactive summary display comprises a generally tabular presentation, the presentation including a series of rows and columns of information, each row corresponding to a respective one of the data fields of the dataset. 
     
     
         11 . The method of  claim 10 , wherein the tabular presentation includes:
 a first column listing a name of the corresponding field in each row;   a second column listing a corresponding data type in each row; and   a third column listing, in each row, a corresponding number of instances of data that have content in the corresponding data field.   
     
     
         12 . The method of  claim 11 , wherein the tabular presentation includes a fourth row in which a respective histogram is displayed in each row that corresponds to a text field, the histogram presenting in graphical form an indication of a relative number of instances of each token of the corresponding field. 
     
     
         13 . The method of  claim 12 , wherein the histogram comprises a series of vertical bars, wherein each bar represents one of the tokens in the corresponding field, and the relative height of each bar provides a graphic indication of the relative number of instances of each token. 
     
     
         14 . The method of  claim 13 , wherein the presentation implements an interactive feature that, responsive to user selection of one of the bars of a histogram, automatically displays a pop-up panel that shows the corresponding token represented by the selected bar, and the number of occurrences of the corresponding token. 
     
     
         15 . The method of  claim 14 , wherein the presentation includes, adjacent to a text field histogram, a pop-up user control for scrolling the histogram to present additional bars in the display. 
     
     
         16 . The method of  claim 11 , and further comprising, for a selected row (field) of the presentation, generating and displaying a tag cloud representation of the corresponding data. 
     
     
         17 . A visualization method comprising:
 accessing a data model based at least in part on a dataset comprising data items, wherein at least one of the data items includes a text field;   generating a decision tree representation of the data model, wherein the decision tree comprises nodes and branches, wherein at least one of the nodes represents a split based on the content of a text field;   displaying at least a selected portion of the decision tree on an electronic display screen;   displaying a legend along with the displayed portion of the decision tree, the legend indicating each split criteria along the prediction path illustrated by the displayed portion of the decision tree;   wherein, for each text field that appears in the legend, a pop-up panel is provisioned to display additional information, responsive to user selection of a given text field in the legend, the additional information including a word or token used to determine the corresponding split in the decision path.   
     
     
         18 . The method of  claim 17  wherein the additional information further includes a number of occurrences or count of the token in the dataset.