System and Methods of Language Processing

Abstract

The disclosure related to the field of language processing. A server (30) is configured to respond to a query associated with a user device (21) by sending, to the user device (21), an indication of an item selected based on semantic importance attributed to grams of text in the query. Attributing semantic importance comprises: in the event that a number of occurrences of the gram in a first document is above an occurrence threshold, determining a gram score for said gram based on said number of occurrences; in the vent that the number of occurrences of the gram in the first document is below the occurrence threshold, determining the gram score based on: (i) said number of occurrences, and (ii) a reference score for the gram based on a number of occurrences of the gram in a reference document different to the at least one first document; and attributing the semantic importance based on the gram score.

1 . A server comprising:
 a data store storing:   at least one first document, wherein the at least one first document comprises a plurality of item identifiers comprising at least one gram of text; and   an association between each gram and its corresponding gram score; and   a processor coupled to the data store;   wherein the server is configured for natural language processing of text in a query, wherein the query is associated with a user device;   wherein the server is configured to respond to the query by sending, to the user device, an indication of an item selected based on natural language processing of grams of text in the query;   wherein the natural language processing of the text in the query comprises attributing semantic importance to grams of text in the query, wherein attributing semantic importance to the grams of text in the query comprises:   in the event that a number of occurrences of the gram in the at least one first document is above an occurrence threshold, determining a gram score for said gram based on said number of occurrences;   in the event that the number of occurrences of the gram in the at least one first document is below the occurrence threshold, determining the gram score based on: (i) said number of occurrences, and (ii) a reference score for the gram based on a number of occurrences of the gram in at least one reference document different to the at least one first document; and   attributing the semantic importance based on the gram score.   
     
     
         2 . The server of  claim 1  wherein sending, to the user device, the indication of the item comprises providing an output to a resource. 
     
     
         3 . The server of  claim 1  wherein the server is configured to: (i) obtain a new item identifier, and (ii) determine a gram score for each gram of text in the new item identifier. 
     
     
         4 . The server of  claim 3 , where in the processor is configured to update the association based grams in the new item identifier and their corresponding determined gram scores. 
     
     
         5 . The server of  claim 1 , wherein the indication of the item is selected based on semantic importance attributed to grams of text in the plurality of item identifiers in the at least one first document. 
     
     
         6 . The server of  claim 5 , wherein the query comprises a query item identifier; and
 wherein selecting the indication of the item comprises selecting an item identifier, or an indication thereof, based on, for each item identifier from a plurality of item identifiers in the at least one first document, a degree of similarity between the query item identifier and said item identifier.   
     
     
         7 . The server of  claim 1 , wherein the data store comprises a plurality of first documents comprising a plurality of item identifiers;
 wherein the query comprises a plurality of query item identifiers comprising at least one gram of text;   wherein the processor is configured to:   determine a gram score for each gram in each query item identifier;   determine a degree of similarity between each query item identifier and each of a plurality of item identifiers in at least one of the first documents;   select at least one first document based on the determined degree of similarity between each query item identifier and respective item identifiers in each of the at least one first documents; and   send, to the user device an indication of at least one of: (i) the at least one selected first document and (ii) the selected plurality of respective item identifiers in said selected first document.   
     
     
         8 . The server of  claim 7  wherein each of a plurality of the plurality of first documents is associated with a corresponding facility;
 wherein the server is configured to receive an acceptance message from the user device in response to the at least one item identifier sent to the user device; and 
 wherein the acceptance message comprises an indication of whether or not the selected plurality of respective item identifiers for one selected first document are approved. 
 
     
     
         9 . The server of  claim 8 , wherein, in the event that the acceptance message comprises an approval of the selected plurality of respective item identifiers from the selected first document, the server is configured to send an indication of said plurality of respective item identifiers to the facility corresponding to said first document. 
     
     
         10 . The server of  claim 1  wherein in the event that the reference score for a gram of text is greater than a reference threshold value, the processor is configured to determine a lower gram score for said gram. 
     
     
         11 . The server of  claim 10 , wherein the lower gram score is determined based on the reference score. 
     
     
         12 . The server of  claim 1  wherein determining the gram score for a gram of text comprises reducing the value of the gram score in the event that an original value for the gram score would be larger than a gram threshold. 
     
     
         13 . The server of  claim 12 , wherein the processor is configured to determine the size of the reduction to the gram score based on a difference between the original value for the gram score and the gram threshold. 
     
     
         14 . The server of  claim 1  wherein the data store comprises a plurality of trigger grams, wherein the processor is configured to detect a trigger gram in an item identifier and to alter the gram score of any subsequent grams in the item identifier. 
     
     
         15 . The server of  claim 14 , wherein altering the gram score comprises at least one of: reducing the gram score and separating the item identifier into two portions, a first portion for the grams before the trigger gram and a second portion for the grams after the trigger gram. 
     
     
         16 . The server of  claim 1  wherein the number of occurrences for a gram in the at least one first document comprises a total number of item identifiers in the at least one first document which include said gram. 
     
     
         17 . A computer-implemented method of responding to a query associated with a user device, the method comprising:
 obtaining a query associated with a user device;   performing natural language processing of text in the query by attributing semantic importance to a gram of text in the query, wherein attributing semantic importance comprises:   in the event that a number of occurrences of the gram in at least one first document is above an occurrence threshold, determining a gram score for said gram based on said number of occurrences;   in the event that the number of occurrences of the gram in the at least one first document is below the occurrence threshold, determining the gram score based on: (i) said number of occurrences, and (ii) a reference score for the gram based on a number of occurrences of the gram in at least one reference document different to the at least one first document; and   attributing the semantic importance based on the gram score;   selecting an indication of an item based on the semantic importance attributed to grams of text in the query; and   sending, to the user device associated with the query, an indication of the selected item.   
     
     
         18 . A computer-implemented method for responding to a query, the method comprising:
 obtaining a query comprising an item identifier relating to a first corpus of text;   performing natural language processing of text in the query by attributing semantic importance to grams of text in the query, wherein attributing semantic importance comprises:   in the event that a term frequency for the gram in the first corpus of text is above a threshold value, determining a gram score for said gram based on said term frequency;   in the event that said term frequency is below the threshold value, determining the gram score based on a term frequency for the gram in a second unrelated corpus of text; and   attributing the semantic importance based on the gram score;   responding to the query by providing output to a resource, wherein the output is selected based on the semantic importance attributed to grams in the query.   
     
     
         19 . A computer program product comprising program instructions configured to program a processor to perform the method of  claim 17 . 
     
     
         20 . A computer program product comprising program instructions configured to program a processor to perform the method of  claim 18 .