Method and subsystem for information acquisition and aggregation to facilitate ontology and language model generation within a content-search-service system

Abstract

Various embodiments of the present invention include information-aggregation-and-classification components of content-search-service systems which acquire information from information sources, aggregate and normalize the acquired information, and classify the acquired information prior to storing the normalized and classified information for use by language-model-builder components and ontology-builder components of the content-search-service systems. Additional embodiments of the present invention include the ontology-builder components, which builds ontologies from the normalized and classified information for specific dates, date/times, date ranges, or date/time ranges and for specific categories.

1 . (canceled) 
     
     
         2 . A method for classifying information from an information source, the method comprising:
 collecting, with a computer, raw information from the information source, wherein the raw information includes words and non-text items;   storing the raw information;   processing the raw information to create processed information, wherein the processing includes:   removing the non-text items from the raw information;   normalizing the words of the raw information; and   determining a number of occurrences for a plurality of words of the raw information;   generating at least one term vector comprising:   a first word value indicative of a first word of the plurality of words;   a first occurrence value indicative of the number of occurrences of the first word;   a second word value indicative of a second word of the plurality of words; and   a second occurrence value indicative of the number of occurrences of the second word; and   determining a category, from a plurality of categories, with which to associate the processed information, wherein the category is determined using a probability of, for at least one word of the at least one term vector, whether either or both of the word value and the occurrence value are in a document associated with the category.   
     
     
         3 . The method for classifying the information from the information source recited in  claim 2 , wherein the normalizing the words of the raw information includes at least one item selected from the group consisting of:
 changing upper-case characters of each word to lower case,   changing a plural form of a word to a singular form of the word, and   replacing a derived word with a root of the derived word.   
     
     
         4 . The method for classifying the information from the information source recited in  claim 1 , wherein the processing further comprises removing words from the raw information that are not nouns. 
     
     
         5 . The method for classifying the information from the information source recited in  claim 2 , wherein the processing further comprises removing words from the raw information that occur on a list. 
     
     
         6 . The method for classifying the information from the information source recited in  claim 2 , wherein the information source is a web page. 
     
     
         7 . The method for classifying the information from the information source recited in  claim 6 , wherein the web page comprises a first web page, further comprising determining, with the computer, the first web page from a link of a second web page. 
     
     
         8 . The method for classifying the information from the information source recited in  claim 2 , wherein the probability of whether either or both of the word value and the occurrence value are in a document associated with the category is based, at least in part, on quantities determined from previously-observed data. 
     
     
         9 . A computer system for classifying information from an information source, the computer system comprising:
 a network interface;   a memory; and   a processor communicatively coupled with the network interface and the memory, wherein the processor is configured to cause the computer system to:   collect, using the network interface, raw information from the information source, wherein the raw information includes words and non-text items;   store the raw information in the memory;   process the raw information to create processed information by:   removing the non-text items from the raw information;   normalizing the words of the raw information; and   determining a number of occurrences for a plurality of words of the raw information;   generate at least one term vector comprising:   a first word value indicative of a first word of the plurality of words;   a first occurrence value indicative of the number of occurrences of the first word;   a second word value indicative of a second word of the plurality of words; and   a second occurrence value indicative of the number of occurrences of the second word; and   determine a category, from a plurality of categories, with which to associate the processed information, wherein the category is determined using a probability of, for at least one word of the at least one term vector, whether either or both of the word value and the occurrence value are in a document associated with the category.   
     
     
         10 . The computer system for classifying the information from the information source recited in  claim 9 , wherein the normalizing the words of the raw information includes at least one item selected from the group consisting of:
 changing upper-case characters of each word to lower case,   changing a plural form of a word to a singular form of the word, and   replacing a derived word with a root of the derived word.   
     
     
         11 . The computer system for classifying the information from the information source recited in  claim 9 , wherein the processor is configured to cause the computer system to further process the raw information by removing words from the raw information that are not nouns. 
     
     
         12 . The computer system for classifying the information from the information source recited in  claim 9 , wherein the processor is configured to cause the computer system to further process the raw information by removing words from the raw information that occur on a list. 
     
     
         13 . The computer system for classifying the information from the information source recited in  claim 9 , wherein the processor is configured to cause the computer system to access a web page comprising the information source. 
     
     
         14 . The computer system for classifying the information from the information source recited in  claim 13 , wherein:
 the web page comprises a first web page; and   the processor is configured to further cause the computer system to determine the first web page from a link of a second web page.   
     
     
         15 . The computer system for classifying the information from the information source recited in  claim 9 , wherein the processor is configured to further cause the computer system to determine the probability of whether either or both of the word value and the occurrence value are in a document associated with the category based, at least in part, on quantities determined from previously-observed data. 
     
     
         16 . A non-transitory machine-readable medium for classifying the information from the information source, the medium having instructions embedded thereon which, when executed by one or more machines, cause the one or more machines to:
 collect raw information from the information source, wherein the raw information includes words and non-text items;   store the raw information in the memory;   process the raw information to create processed information by:   removing the non-text items from the raw information;   normalizing the words of the raw information; and   determining a number of occurrences for a plurality of words of the raw information;   generate at least one term vector comprising:   a first word value indicative of a first word of the plurality of words;   a first occurrence value indicative of the number of occurrences of the first word;   a second word value indicative of a second word of the plurality of words; and   a second occurrence value indicative of the number of occurrences of the second word; and   determine a category, from a plurality of categories, with which to associate the processed information, wherein the category is determined using a probability of, for at least one word of the at least one term vector, whether either or both of the word value and the occurrence value are in a document associated with the category.   
     
     
         17 . The non-transitory machine-readable medium for classifying the information from the information source recited in  claim 16 , wherein the normalizing the words of the raw information includes at least one item selected from the group consisting of:
 changing upper-case characters of each word to lower case,   changing a plural form of a word to a singular form of the word, and   replacing a derived word with a root of the derived word.   
     
     
         18 . The non-transitory machine-readable medium for classifying the information from the information source recited in  claim 16 , wherein the instructions, when executed by the one or more machines, cause the one or more machines to further process the raw information by performing at least one of:
 removing words from the raw information that are not nouns; or   removing words from the raw information that occur on a list.   
     
     
         19 . The non-transitory machine-readable medium for classifying the information from the information source recited in  claim 16 , wherein the instructions, when executed by the one or more machines, cause the one or more machines to further access a web page comprising the information source. 
     
     
         20 . The non-transitory machine-readable medium for classifying the information from the information source recited in  claim 19 , wherein:
 the web page comprises a first web page; and   the instructions, when executed by the one or more machines, further cause the one or more machines to determine the first web page from a link of a second web page.   
     
     
         21 . The non-transitory machine-readable medium for classifying the information from the information source recited in  claim 16 , wherein the instructions, when executed by the one or more machines, further cause the one or more machines to determine the probability of whether either or both of the word value and the occurrence value are in a document associated with the category based, at least in part, on quantities determined from previously-observed data.