Document retrieval using internal dictionary-hierarchies to adjust per-subject match results

Abstract

Techniques for managing big data include retrieval using per-subject dictionaries having and hierarchical information indicating affinity between the dictionaries. A retrieval request for one or more documents pertaining to one or more subjects is performed by specifying the subjects, selecting dictionaries associated with the subjects, matching the subjects to documents in the collection according to a strength of relationship between the documents and the subjects and also using the hierarchical information, to obtain a subset of the collection of documents.

What is claimed is: 
     
         1 . A computer-performed method of retrieving documents pertaining to one or more subjects from a collection of documents, the method comprising:
 within a computer system, specifying the one or more subjects;   by at least one processor of the computer system, selecting dictionaries from a set of dictionaries uniquely associated with corresponding ones of the one or more subjects;   by the at least one processor, matching the one or more subjects to documents in the collection of documents according to strength of a relationship between the documents and the one or more subjects and hierarchical information that indicates an affinity between the selected dictionaries, to obtain a subset of the collection of documents relevant to the one or more subjects; and   returning, by the at least one processor, the subset of the collection of documents.   
     
     
         2 . The method of  claim 1 , wherein the strength of the relationship between the documents and the one or more subjects is determined from entries within the dictionaries having associated subject-determining power scores corresponding to descriptive terms within the entries, wherein an individual subject-determining-power score indicates the relative strength or weakness of the corresponding descriptive term with respect to the subject associated with a particular dictionary containing the entry, and wherein the matching generates the document scores using the subject-determining power scores. 
     
     
         3 . The method of  claim 2 , wherein the adjusting adjusts an order of the subset of documents returned by the matching according to the change in document scores. 
     
     
         4 . The method of  claim 2 , wherein the adjusting is performed by multiplying the subject-determining power scores by a factor of (1+α/D), where α is a tuning parameter and D is a distance indicating a degree of decreasing affinity within the hierarchy of the corresponding dictionary. 
     
     
         5 . The method of  claim 1 , further comprising computing, by the at least one processor, the affinity values according to a traversal distance between pairs of nodes of the tree structure corresponding to pairs of the terms or a difference in hierarchical levels of the terms within the tree structure. 
     
     
         6 . The method of  claim 1 , wherein the hierarchical information is organized as a tree structure having hierarchical levels corresponding to levels of classification, wherein a highest level of the hierarchy of classification is the subject associated with the dictionary and wherein the lowest level of the hierarchy of classification contains descriptive terms that do not describe a further sub-classification, with terms corresponding to sub-classifications of the subject disposed between the highest level of the hierarchy of classification and the lowest level of the hierarchy of classification. 
     
     
         7 . A computer system comprising a processor for executing program instructions coupled to a memory for storing the program instructions, wherein the program instructions are program instructions for retrieving documents pertaining to one or more subjects from a collection of documents, and wherein the program instructions comprise:
 program instructions that specify the one or more subjects;   program instructions that select from a set of dictionaries uniquely associated with corresponding ones of the one or more subjects;   program instructions that match the one or more subjects to documents in the collection of documents according to strength of a relationship between the documents and the one or more subjects and hierarchical information that indicates an affinity between the selected dictionaries, to obtain a subset of the collection of documents relevant to the one or more subjects; and   program instructions that return the subset of the collection of documents obtained by the matching.   
     
     
         8 . The computer system of  claim 7 , wherein the strength of the relationship between the documents and the one or more subjects is determined from entries within the dictionaries having associated subject-determining power scores corresponding to descriptive terms within the entries, wherein an individual subject-determining-power score indicates the relative strength or weakness of the corresponding descriptive term with respect to the subject associated with a particular dictionary containing the entry, and wherein the matching generates the document scores using the subject-determining power scores. 
     
     
         9 . The computer system of  claim 8 , wherein the program instructions that adjust change an order of the subset of documents returned by the program instructions that match according to the change in document scores. 
     
     
         10 . The computer system of  claim 8 , wherein the program instructions for that adjust multiply the subject-determining power scores by a factor of (1+α/D), where α is a tuning parameter and D is a distance indicating a degree of decreasing affinity within the hierarchy of the corresponding dictionary. 
     
     
         11 . The computer system of  claim 7 , further comprising program instructions that compute the affinity values according to a traversal distance between pairs of nodes of the tree structure corresponding to pairs of the terms or a difference in hierarchical levels of the terms within the tree structure. 
     
     
         12 . The computer system of  claim 7  wherein the hierarchical information is organized as a tree structure having hierarchical levels corresponding to levels of classification, wherein a highest level of the hierarchy of classification is the subject associated with the dictionary and wherein the lowest level of the hierarchy of classification contains descriptive terms that do not describe a further sub-classification, with terms corresponding to sub-classifications of the subject disposed between the highest level of the hierarchy of classification and the lowest level of the hierarchy of classification. 
     
     
         13 . A computer program product comprising a computer-readable storage device storing program instructions for retrieving documents pertaining to one or more subjects from a collection of documents, wherein the program instructions comprise program instructions for:
 specifying the one or more subjects;   selecting dictionaries from a set of dictionaries uniquely associated with corresponding ones of the one or more subjects;   matching the one or more subjects to documents in the collection of documents according to strength of a relationship between the documents and the one or more subjects and hierarchical information that indicates an affinity between the selected dictionaries, to obtain a subset of the collection of documents relevant to the one or more subjects; and   returning, by at least one processor, the subset of the collection of documents.   
     
     
         14 . The computer program product of  claim 13 , wherein the strength of the relationship between the documents and the one or more subjects is determined from entries within the dictionaries having associated subject-determining power scores corresponding to descriptive terms within the entries, wherein an individual subject-determining-power score indicates the relative strength or weakness of the corresponding descriptive term with respect to the subject associated with a particular dictionary containing the entry, and wherein the matching generates the document scores using the subject-determining power scores. 
     
     
         15 . The computer program product of  claim 14 , wherein the program instructions for adjusting change an order of the subset of documents returned by the program instructions for second matching according to the change in document scores. 
     
     
         16 . The computer program product of  claim 14 , wherein the program instructions for adjusting multiply the subject-determining power scores by a factor of (1+α/D), where α is a tuning parameter and D is a distance indicating a degree of decreasing affinity within the hierarchy of the corresponding dictionary. 
     
     
         17 . The computer program product of  claim 13 , further comprising program instructions for computing the affinity values according to a traversal distance between pairs of nodes of the tree structure corresponding to pairs of the terms or a difference in hierarchical levels of the terms within the tree structure. 
     
     
         18 . The computer program product of  claim 13 , wherein the hierarchical information is organized as a tree structure having hierarchical levels corresponding to levels of classification, wherein a highest level of the hierarchy of classification is the subject associated with the dictionary and wherein the lowest level of the hierarchy of classification contains descriptive terms that do not describe a further sub-classification, with terms corresponding to sub-classifications of the subject disposed between the highest level of the hierarchy of classification and the lowest level of the hierarchy of classification.