Machine Learning Systems and Methods for Predicting Personal Information Using File Metadata

Abstract

Systems, methods and apparatuses are disclosed to efficiently and accurately scan a plurality of documents located in any number of unstructured data sources. Preprocessed metadata is generated for each document and metadata features are determined based on the preprocessed metadata. A trained machine learning system may utilize the metadata features to predict whether each of the documents contains personal information, without requiring any information relating to the content of such documents.

What is claimed is: 
     
         1 . A system for predicting personal information presence in unstructured data, the system comprising:
 a data source storing a plurality of documents, each document associated with unstructured data and metadata information comprising a plurality of metadata items, the metadata items comprising a document path and a document name;   a trained machine learning system that has previously been trained with training data to determine personal information predictions based only on metadata features of data records, each personal information prediction relating to a probability that a document associated with a given data record contains personal information; and   a server in communication with the data source and the trained machine learning system, the server configured to:
 receive, for each of the documents, the respective metadata information; 
 preprocess, for each of the documents, the respective metadata information to generate preprocessed metadata, said preprocessing comprising:
 for each metadata item of the plurality of metadata items:
 normalizing the metadata item to generate a normalized metadata item; and 
 tokenizing the normalized metadata item to generate a tokenized metadata item comprising a plurality of tokens; 
 
 
 create, for each of the documents, metadata features based on the respective preprocessed metadata, said creating comprising:
 calculating, for each of the tokenized metadata items, a total number of occurrences of each token of the plurality of tokens; 
 
 create, for each of the documents, a data record comprising the respective metadata features; 
 provide the data records to the trained machine learning system; 
 receive, from the trained machine learning system, a personal information prediction for each of the documents; and 
 provide the personal information predictions to a user. 
   
     
     
         2 . A system according to  claim 1 , wherein said normalizing comprises removing one or more predefined characters from the metadata item. 
     
     
         3 . A system according to  claim 1 , wherein said normalizing comprises transforming the metadata item to lowercase. 
     
     
         4 . A system according to  claim 1 , wherein said tokenizing comprises inferring positions of spaces and splitting the normalized metadata into words at each of the inferred positions. 
     
     
         5 . A system according to  claim 4 , wherein said inferring comprises:
 creating a plurality of potential word combinations from the metadata item, each potential word combination comprising only words for which an entry exists in a dictionary;   determining a total cost of each of the potential word combinations; and   selecting an optimal word combination from the potential word combinations based on a comparison of the determined total costs.   
     
     
         6 . A system according to  claim 5 , wherein the dictionary comprises a plurality of entries, each entry comprising a valid word associated with a relative frequency of use. 
     
     
         7 . A system according to  claim 4 , wherein said tokenizing further comprises: removing one or more predefined words from the words and/or reducing at least one of the words to its respective root. 
     
     
         8 . A system according to  claim 1 , wherein the metadata items further comprise at least one of: a document description, a document title and document keywords. 
     
     
         9 . A system according to  claim 1 , wherein the metadata features for each of the documents further comprise a feature relating to a document creation date, a document modification date, a document size, a document type, a document author and/or a document owner. 
     
     
         10 . A system according to  claim 1 , wherein the server and/or the trained machine learning system is further configured to:
 determine that the probability associated with the personal information prediction relating to a document is greater than a predetermined threshold; and   classify the document as containing personal information, based on the said determining.   
     
     
         11 . A system according to  claim 1 , wherein each of the personal information predictions relates to a probability that a document associated with a given data record contains a particular personal information attribute. 
     
     
         12 . A system according to  claim 11 , wherein the particular personal information attribute is selected from the group consisting of: an email address, an IP address, a social security number and a zip code.