Method of thematic classification of documents, themetic classification module, and search engine incorporating such a module

Abstract

A method of thematically classifying documents, in particular for making up or updating thematic databases ( 42 ) for a search engine, includes the steps of selecting documents representative of each theme, identifying within the selected documents, elements that are characteristic of each theme, allocating a coefficient (R) to each identified element, said coefficient being representative of the relevance of said element relative to the corresponding theme, and for each document ( 50 ) for classification, identifying said elements characteristic of each theme contained in the document and, for each theme corresponding thereto, using the coefficients allocated to said elements to calculate the value of a characteristic representative of the relevance of the theme for the document ( 50 ), in order to decide whether or not the document relates to the theme.

1. A method of thematically classifying documents, in particular for making up or updating thematic databases for a search engine, the method comprising the following steps:
 manually and/or automatically selecting a sample of documents representative of each theme; 
 automatically identifying within the selected documents elements that are characteristic of each said theme; 
 automatically allocating a coefficient to each identified element, wherein said coefficient is representative of a relevance of said element to a corresponding theme; 
 downloading documents from a computer network; 
 for each downloaded document to be classified, identifying said theme-characterizing elements that are contained in the document for each said theme, and for each theme corresponding to the elements, using the coefficients allocated to said elements to calculate a characteristic value representative of the relevance of that theme for the document, in order to decide whether or not the document relates to the theme, said theme—characterizing elements identification and calculation steps being performed automatically for each document downloaded from the computer network; 
 automatically classifying the downloaded documents as a function of themes with which they deal; 
 automatically storing the documents classified thematically in databases that can be interrogated on the basis of themes contained in a request; and 
 making the databases available to users who interrogate the databases on the basis of themes contained in a request; 
 and the step of allocating said coefficient to each identified element comprises the following steps for each theme: 
 automatically calculating a frequency of the element in the selected documents relating to the theme; 
 automatically calculating a frequency of the element in the selected documents that do not relate to the theme; and 
 automatically calculating a ratio of the calculated frequencies of the theme-related element and of the non-theme-related element. 
 
   
   
     2. A method according to  claim 1 , further comprising the step of automatically sorting themes in a theme tree structure in decreasing order of coefficients. 
   
   
     3. A method according to  claim 1 , wherein the step of automatically calculating the characteristic value representative of the relevance of the theme of a document for classification comprises the following steps, for each theme:
 reading the value of the ratio of said frequencies for each theme-representing element extracted from the document; 
 multiplying together the values of the ratios as read for the theme-characterizing elements; and 
 allocating the result of the multiplication to the characteristic value. 
 
   
   
     4. A method according to  claim 1 , wherein it is automatically decided that the document relates to a theme if the characteristic value of representative of the relevance of the theme for said document is greater than a threshold value. 
   
   
     5. A method according to  claim 4 , wherein the threshold value for each theme is automatically determined on the basis of the frequency ratio using the following relationship:
   score−threshold theme =( R   mean )theme  n    
 in which: 
 score−threshold theme  designates the threshold value; 
 R mean  represents the mean value of frequency ratios R of the elements of the theme; and 
 theme −n designates a predetermined number. 
 
   
   
     6. A method according to  claim 4 , wherein the threshold value is adjusted manually. 
   
   
     7. A method according to  claim 1 , wherein the steps of automatically identifying theme-characterizing elements contained in a document and for each theme are performed by means of a hashing table. 
   
   
     8. A method according to  claim 1 , wherein for each vocabulary element of a request formulated by a user, coefficients are automatically calculated that are characteristic of the element relative to each known theme, and each element is associated with the corresponding themes and coefficients, so that said coefficients reach a minimum value. 
   
   
     9. A module for thematically classifying documents, in particular for a search engine, the module comprising a central processor unit having means for comparing elements extracted from each document with elements characteristic of various themes, each element being allocated a coefficient representative of a relevance of said element for a corresponding theme, and means for calculating a characteristic value representative of the relevance of a theme for the document on the basis of the coefficients of said characteristic elements that the document contains, in order to decide whether or not the document relates to said theme, said central processor unit being coupled to means for storing documents classified by theme that can be interrogated on the basis of themes contained in a request, and the module has means for calculating a frequency of the element in the documents relating to the theme, means for calculating a frequency of the element in the documents that do not relate to the theme, and means for calculating a ratio between the calculated frequencies of the theme-related element and of the non-theme-related element. 
   
   
     10. A method of classifying documents comprising the steps of providing the module of  claim 9 , and using said module to determine which themes are contained in a request formulated by a user. 
   
   
     11. A method of for thematically classifying documents comprising the steps of providing the module of  claim 9 , using said module for determining which themes are contained in pages downloaded from a computer network or in a request formulated by a user, and filtering downloaded documents to ban consultation of pages relating to one or more predetermined themes. 
   
   
     12. A method of thematically classifying documents comprising the steps of providing the module of to  claim 9 , using said module to determine which themes are contained in a request formulated by a user, and generating user profiles on the basis of the themes to which the request relates. 
   
   
     13. A search engine for documents on a computer network, the engine comprising an indexing module for creating and updating thematic databases on the basis of documents downloaded from the computer network, and a module for interrogating thematic databases adapted to supply references of documents corresponding to a request that has been input thereto, the search engine further comprising a thematic classification module according to  claim 9  associated with the indexing module.