System and method for detecting meaningless lexical units in a text of a message

Abstract

There is provided a system and a method for detecting meaningless lexical units in a text message, the method comprising: performing a syntax analysis of the text message and determining at least one lexical unit as a first potential meaningless lexical unit; determining a control sum of the first potential meaningless lexical unit; matching using a first parameter, the first potential meaningless lexical unit with lexical units from a plurality of lexical units from a lexical units database, wherein matching using the first parameter comprises matching a control sum of the first potential meaningless lexical unit with control sums of meaningless lexical units from the lexical units database; determining the first potential meaningless lexical unit is a meaningless lexical unit if the lexical units database includes at least one meaningless lexical unit with a control sum corresponding to the control sum of the first potential meaningless lexical unit.

The invention claimed is: 
     
       1. A computer implemented method for detecting meaningless lexical units in an electronic message received by a server as to generate an abstract of the electronic message, the method comprising:
 (i) performing, by the server, a syntax analysis of a most significant part of the electronic message and determining at least one lexical unit as a first potential meaningless lexical unit, the first potential meaningless lexical unit comprising a plurality of control elements, the most significant part of the electronic message having been determined by analyzing a most significant logical block of source code from a plurality of the logical blocks of a source code of the electronic message; 
 (ii) determining, by the server, a numerical control sum of the first potential meaningless lexical unit, the numerical control sum being based on the plurality of control elements having a respective numeric value representative of the first potential meaningless lexical unit; 
 (iii) using the numerical control sum, accessing a lexical unit database located on the server, the lexical unit database containing a plurality of pre-determined meaningless lexical units with their associated pre-determined numerical control sums; 
 (iv) determining, by the server, the first potential meaningless lexical unit is a meaningless lexical unit if the lexical units database includes at least one meaningless lexical unit with the pre-determined numerical control sum matching the numerical control sum of the first potential meaningless lexical unit, the matching comprising checking a measure of a difference between the numerical control sums and determining the numerical control sums as matching if the measure of the difference is within a predefined permissible amplitude of the difference; 
 (v) if the first potential meaningless lexical unit is determined to be the meaningless lexical unit, generating, by the server, the abstract of the electronic message, the abstract not including the at least one meaningless lexical unit. 
 
     
     
       2. The method of  claim 1 , if there is no meaningless lexical unit with a predetermined numerical control sum matching the numerical control sum of the first potential meaningless lexical unit in the lexical unit database, the method further comprises:
 (i) subdividing the first potential meaningless lexical unit into at least two smaller lexical units and determining at least one smaller lexical unit as the second potential meaningless lexical unit; 
 (ii) determining, by the server, a numerical control sum of the second potential meaningless lexical unit; 
 (iii) matching using a second parameter the first potential meaningless lexical unit with lexical units from the lexical units database, wherein matching using the second parameter comprises matching the numerical control sum of the second potential meaningless lexical unit with predetermined numerical control sums of meaningless lexical units from the lexical units database; 
 (iv) determining, by the server, the second potential meaningless lexical unit is a meaningless lexical unit if the lexical units database includes at least one meaningless lexical unit with a predetermined numerical control sum matching the numerical control sum of the second potential meaningless lexical unit; 
 (v) if the second potential meaningless lexical unit is determined to be the meaningless lexical unit, generating, by the server, the abstract of the electronic message, the abstract not including the at least one meaningless lexical unit. 
 
     
     
       3. The method of  claim 2 , wherein the first potential meaningless lexical unit is a paragraph and the second potential meaningless lexical unit is a sentence from the paragraph. 
     
     
       4. The method of  claim 1 , wherein the plurality of control elements comprises at least one of: a number of characters in a lexical unit, a number of letters in the lexical unit, a number of capital letters in the lexical unit, a number of lower-case letters in the lexical unit, a number of spaces in the lexical unit, a number of numbers in the lexical unit, a number of special marks, a number of words in the lexical unit, a size of the lexical unit expressed in information handling and storage units. 
     
     
       5. The method of  claim 4 , wherein the matching using the first parameter is carried out using a first set of control elements and matching using the second parameter is carried out using a second set of control elements. 
     
     
       6. The method of  claim 5 , wherein the first set of control elements and the second set of control elements are the same. 
     
     
       7. The method of  claim 1 , wherein a lexical database includes at least one meaningless lexical unit with a numerical control sum corresponding to the numerical control sum of the potential meaningless lexical unit, the method further comprises performing a character-by-character match of the potential meaningless lexical unit with the at least one meaningless lexical unit and wherein in response to a match of a character sequence of the potential meaningless lexical unit with a character sequence of the at least one meaningless lexical unit, the method further comprises determining the potential meaningless lexical unit as a meaningless lexical unit. 
     
     
       8. The method of  claim 1 , wherein a lexical unit from the plurality of lexical units from the lexical unit database is meaningless if its weight exceeds the predefined threshold value. 
     
     
       9. The method of  claim 8 , wherein the lexical unit database is generated on the basis of the plurality of lexical units which can be found in the plurality of text messages and in which a weight of each lexical unit is in direct proportion with the given lexical unit frequency in the plurality of lexical units which can be found in the plurality of the text messages. 
     
     
       10. The method of  claim 1 , wherein analyzing the markup language source code of the electronic message comprises analyzing at least one of: a structure of a electronic message, a font type, a font size, a font face, punctuation marks, and special marks. 
     
     
       11. The method of  claim 1 , wherein carrying out the syntax analysis of the electronic message comprises executing a syntax analysis of a predefined number of paragraphs from the beginning of the electronic message. 
     
     
       12. The method of  claim 1 , wherein the electronic message is an e-mail message. 
     
     
       13. The method of  claim 1 , wherein the most significant logical block of the source code comprises a block of the source code which comprises text, and the size of which is larger than a size of any other logical block of the source code of the e-mail message. 
     
     
       14. The method of  claim 1 , wherein the most significant logical block of the source code is a block of the source code which comprises text, and wherein the text of the most significant logical block of the source code comprises the majority of meaningful lexical units in comparison with the text of any other logical block of the source code of the given e-mail message. 
     
     
       15. The method of  claim 1 , wherein the lexical unit is any of:
 a word, 
 a phrase, 
 a sentence, 
 a paragraph. 
 
     
     
       16. The method of  claim 1 , wherein determining at least one lexical unit as a potential meaningless lexical unit comprises determining at least one meaningful lexical unit. 
     
     
       17. The method of  claim 1 , wherein determining at least one lexical unit as a potential meaningless lexical unit is performed on the basis of the syntax analysis of one of:
 an entire text of the electronic message, and 
 a part of the text of the electronic message. 
 
     
     
       18. The method of  claim 1 , wherein an unique control sum is an ID of a unique lexical unit. 
     
     
       19. A computer including a processor, the processor being configured to render the computer operable to execute:
 (i) performing a syntax analysis of a most significant part of an electronic message and determining at least one lexical unit as a first potential meaningless lexical unit, the first potential meaningless lexical unit comprising a plurality of control elements, the most significant part of the electronic message having been determined by analyzing a most significant logical block of source code from a plurality of the logical blocks of a source code of the electronic message; 
 (ii) determining a numerical control sum of the first potential meaningless lexical unit, the control sum being based on the plurality of control elements having a respective numeric value representative of the first potential meaningless lexical unit; 
 (iii) using the numerical control sum, accessing a lexical unit database located on the server, the lexical unit database containing a plurality of pre-determined meaningless lexical units with their associated pre-determined numerical control sums; 
 (iv) determining the first potential meaningless lexical unit is a meaningless lexical unit if the lexical units database includes at least one meaningless lexical unit with the pre-determined numerical control sum matching the numerical control sum of the first potential meaningless lexical unit, the matching comprising checking a measure of a difference between the numerical control sums and determining the numerical control sums as matching if the measure of the difference is within a predefined permissible amplitude of the difference; 
 (v) if the first potential meaningless lexical unit is determined to be the meaningless lexical unit, generating, by the server, the abstract of the electronic message, the abstract not including the at least one meaningless lexical unit. 
 
     
     
       20. The computer of  claim 19 , if there is no meaningless lexical unit with a predetermined numerical control sum corresponding to the numerical control sum of the first potential meaningless lexical unit in the lexical unit database, the processor being further configured to render the computer operable to execute:
 (i) subdividing the first potential meaningless lexical unit into at least two smaller lexical units and determining at least one smaller lexical unit as the second potential meaningless lexical unit; 
 (ii) determining a numerical control sum of the second potential meaningless lexical unit; 
 (iii) matching using a second parameter the first potential meaningless lexical unit with lexical units from the lexical units database, wherein matching using the second parameter comprises matching the numerical control sum of the second potential meaningless lexical unit with predetermined numerical control sums of meaningless lexical units from the lexical units database; 
 (iv) determining the second potential meaningless lexical unit is a meaningless lexical unit if the lexical units database includes at least one meaningless lexical unit with a predetermined numerical control sum matching the numerical control sum of the second potential meaningless lexical unit; 
 (v) if the second potential meaningless lexical unit is determined to be the meaningless lexical unit, generating, by the server, the abstract of the electronic message, the abstract not including the at least one meaningless lexical unit.