Methods and systems to fingerprint textual information using word runs

Abstract

The present invention provides methods and systems to enable fast, efficient, and scalable means for fingerprinting textual information using word runs. The present system receives textual information and provides algorithms to convert the information into representative fingerprints. In one embodiment, the fingerprints are recorded in a repository to maintain a database of an organization's secure data. In another embodiment, textual information entered by a user is verified against the repository of fingerprints to prevent unauthorized disclosure of secure data. This invention provides approaches to allow derivative works (e.g., different ordering of words, substitution of words with synonyms, etc.) of the original information to be detected at the sentence level or even at the paragraph level. This invention also provides methods and systems for enhancing storage and resource efficiencies by providing approaches to optimize the number of fingerprints generated for the textual information.

We claim: 
     
         1 . A system to prevent unauthorized disclosure of secure information, the system comprising:
 a processor;   a memory;   a processing component configured to:
 receive information including a first text, wherein the first text includes a plurality of words; 
 normalize the first text into a first canonical text expression, the first canonical text expression including a plurality of normalized words; 
 generate a first word hash list for the first canonical text expression, wherein the first word hash list is generated at a word level; and 
 generate one or more fingerprints for the first word hash list, wherein the generation of one or more fingerprints includes:
 assigning a sliding window of size W, wherein W specifies a number of word-value hashes to read from the first word hash list; 
 using the sliding window to read the W word-value hashes from the first word hash list; 
 designating an anchor word-value hash for the sliding window by selecting a distinct-valued word-value hash among the W word-value hashes; and 
 applying a fingerprint hash function to all words starting from a first word-value hash to the anchor word value-hash, wherein applying the fingerprint hash function generates the one or more fingerprints. 
 
   
     
     
         2 . The system to prevent unauthorized disclosure of secure information as recited in  claim 1 , wherein each of the plurality of words being a combination of one or more text characters not separated by a predefined character, and each of the plurality of words being separated from a previous word and a subsequent word by at least one of said predefined character. 
     
     
         3 . A system to prevent unauthorized disclosure of secure information as recited in  claim 2 , wherein said predefined character includes more than one character type, including a space, a period, a comma, a semi-colon, a colon, an exclamation point, a dash, a parenthesis, and a quotation mark. 
     
     
         4 . A system to prevent unauthorized disclosure of secure information as recited in  claim 1 , wherein the processing component is further configured to detect among the plurality of words a separation of one word from a following or preceding word using a word boundary detector. 
     
     
         5 . A system to prevent unauthorized disclosure of secure information as recited in  claim 4 , wherein the word boundary detector is language independent. 
     
     
         6 . A system to prevent unauthorized disclosure of secure information as recited in  claim 1 , wherein said receiving information includes receiving secure information from a local database. 
     
     
         7 . A system to prevent unauthorized disclosure of secure information as recited in  claim 1 , wherein said receiving information includes receiving information entered by a user. 
     
     
         8 . A system to prevent unauthorized disclosure of secure information as recited in  claim 1 , wherein said generating the first word hash list includes converting the plurality of normalized words into a plurality of word-value hashes, wherein each word-value hash of the plurality of word-value hashes represents a specific normalized word. 
     
     
         9 . A system to prevent unauthorized disclosure of secure information as recited in  claim 8 , wherein said converting the plurality of normalized words includes one or more of the following:
 performing case-folding;   removing stop words;   mapping a predefined set of common words to a unique word-value hash;   mapping a predefined set of synonyms to a unique word-value hash;   mapping a predefined set of common words to a unique word-value hash; and   mapping a predefined set of words in a particular category to a unique word-value hash.   
     
     
         10 . A system to prevent unauthorized disclosure of secure information as recited in  claim 1 , wherein the fingerprint hash function includes any hash function that allows the one or more fingerprints to be independent of the order of words in the first word hash list. 
     
     
         11 . A system to prevent unauthorized disclosure of secure information as recited in  claim 1 , wherein the processing component is further configured to record the one or more fingerprints from all sources in a fingerprint database. 
     
     
         12 . A system to prevent unauthorized disclosure of secure information as recited in  claim 1 , wherein the processing component is further configured to monitor and detect any unauthorized disclosure of said secure information by a user, by generating the one or more fingerprints for information entered by the user. 
     
     
         13 . A system to prevent unauthorized disclosure of secure information as recited in  claim 11 , wherein the processing component is further configured to monitor and detect any unauthorized disclosure of said secure information by a user, by generating the one or more fingerprints for information entered by the user and matching the one or more fingerprints against fingerprints stored in said fingerprint database. 
     
     
         14 . A computer implemented method for preventing unauthorized disclosure of secure information, the computer implemented method comprising:
 storing a plurality of secure text fingerprints for a given organization, wherein each of the plurality of secure text fingerprints is generated using a fixed window word run hashing;   receiving a first text that a user desires to transmit outside of the given organization;   generating a first set of fingerprints for the first text using the fixed window word run hashing, wherein generating a first set of fingerprints includes:
 converting a plurality of normalized words into a plurality of word-value hashes to create an original word hash list, wherein each word-value hash represents a specific normalized word; 
 assigning a sliding window of size W, wherein W specifies a number of word-value hashes to read from the original word hash list; 
 using said sliding window to read said W word-value hashes from the original word hash list; 
 designating an anchor word-value hash for the sliding window by selecting a distinct-valued word-value hash among said W word-value hashes; and 
 applying a fingerprint hash function to all words starting from a first word-value hash to the anchor word-value hash, wherein applying the fingerprint hash function generates the first set of fingerprints; 
   determining whether any of the first set of fingerprints is identical to any of the plurality of secure text fingerprints; and   taking a security action when any of the first set of fingerprints is identical to any of the plurality of secure text fingerprints.   
     
     
         15 . A computer implemented method for preventing unauthorized disclosure of secure information as recited in  14 , wherein the fixed window word run hashing comprises:
 receiving information including an original text, the original text including a plurality of words;   normalizing said original text into an original canonical text expression, the original canonical text expression including a plurality of normalized words;   generating an original word hash list for the original canonical text expression, wherein the original word hash list is generated at a word level, wherein the original word hash list includes a plurality of word-value hashes; and   generating an original set of fingerprints for the original word hash list.   
     
     
         16 . A computer implemented method for preventing unauthorized disclosure of secure information as recited in  claim 14 , wherein said first text includes at least one of:
 text contained in an electronic mail;   text contained in a file attached to an electronic mail; and   text that is transferred using a computer's output device.   
     
     
         17 . A computer implemented method for preventing unauthorized disclosure of secure information as recited in  claim 14 , wherein said security action includes at least one of:
 preventing said first text from being transferred;   logging the event as a security violation;   requiring a password from said user to allow said first text to be transferred;   blocking said user's access to said first text; and   sending out a security alert.   
     
     
         18 . A computer implemented method for preventing unauthorized disclosure of secure information as recited in  claim 14 , the computer implemented method further comprising generating the plurality of secure text fingerprints for the given organization. 
     
     
         19 . A computer implemented method for preventing unauthorized disclosure of secure information as recited in  claim 14 , the computer implemented method further comprising creating a fingerprint database for the given organization, wherein the fingerprint database comprises the plurality of secure text fingerprints for the given organization. 
     
     
         20 . A computer implemented method for preventing unauthorized disclosure of secure information as recited in  claim 14 , wherein the fingerprint hash function includes any hash function that allows the first set of fingerprints to be independent of the order of the words in the original word hash list.