Few-shot named-entity recognition

Abstract

A first named entity recognition (NER) system may be adapted to create a second NER system that is able to recognize a new named entity using few-shot learning. The second NER system may process support tokens that provide one or more examples of the new named entity and may process input tokens that may contain the new named entity. The second NER system may use a classifier of the first NER system to compute support token embeddings from the support tokens and input token embeddings from the input tokens. The second NER system may then recognize the new named entity in the input tokens using abstract tag transition probabilities and/or distances between the support token embeddings and the input token embeddings.

What is claimed is: 
     
       1. A computer-implemented method for recognizing named entities, comprising:
 obtaining a classifier for a first named entity recognition task corresponding to a first tag set, wherein the classifier processes tokens to compute score vectors for the tokens; 
 obtaining support tokens corresponding to a second named entity recognition task, wherein the support tokens are labelled with a tag from a second tag set, and wherein the second tag set comprises a first tag that is not in the first tag set; 
 obtaining support token embeddings, wherein the support token embeddings were computed by processing the support tokens with the classifier for the first named entity recognition task; 
 obtaining abstract tag transition probabilities, wherein the abstract tag transition probabilities comprise (i) a transition between a tag that does not correspond to any named entity to a tag that corresponds to any named entity, (ii) a transition between a tag that corresponds to any named entity to a tag that does not correspond to any named entity, and (iii) a transition between a tag that corresponds to any named entity to a tag that corresponds to a different named entity; 
 receiving input tokens for processing with the second named entity recognition task; 
 computing input token embeddings by processing the input tokens with the classifier for the first named entity recognition task; 
 computing distances between the input token embeddings and the support token embeddings; 
 assigning the first tag from the second tag set to a first input token using the distances and the abstract tag transition probabilities; and 
 recognizing a first named entity in the input tokens, wherein the first named entity corresponds to the first tag. 
 
     
     
       2. The computer-implemented method of  claim 1 , wherein the classifier comprises layers that compute a token embedding and one or more output layers. 
     
     
       3. The computer-implemented method of  claim 1 , wherein the first tag set and the second tag set have at least one tag in common. 
     
     
       4. The computer-implemented method of  claim 1 , wherein the support tokens comprise one example of each tag in the second tag set. 
     
     
       5. The computer-implemented method of  claim 1 , wherein the support token embeddings are computed using a value of the classifier prior to one or more output layers. 
     
     
       6. The computer-implemented method of  claim 1 , wherein assigning the first tag from the second tag set to the first input token comprises computing nearest neighbor score vector indicating a match between the first input token and each tag in the second tag set. 
     
     
       7. The computer-implemented method of  claim 6 , wherein assigning the first tag from the second tag set to the first input token comprises processing the nearest neighbor scores and the abstract tag transition probabilities with a Viterbi decoder. 
     
     
       8. A system, comprising:
 at least one server computer comprising at least one processor and at least one memory, the at least one server computer configured to:
 obtain a classifier for a first named entity recognition task corresponding to a first tag set, wherein the classifier processes tokens to compute score vectors for the tokens; 
 obtain support tokens corresponding to a second named entity recognition task, wherein the support tokens are labelled with a tag from a second tag set, and wherein the second tag set comprises a first tag that is not in the first tag set; 
 obtain support token embeddings, wherein the support token embeddings were computed by processing the support tokens with the classifier for the first named entity recognition task; 
 obtain abstract tag transition probabilities, wherein the abstract tag transition probabilities comprise (i) a transition between a tag that does not correspond to any named entity to a tag that corresponds to any named entity, (ii) a transition between a tag that corresponds to any named entity to a tag that does not correspond to any named entity, and (iii) a transition between a tag that corresponds to any named entity to a tag that corresponds to a different named entity; 
 receive input tokens for processing with the second named entity recognition task; 
 compute input token embeddings by processing the input tokens with the classifier for the first named entity recognition task; 
 compute distances between the input token embeddings and the support token embeddings; 
 assign the first tag from the second tag set to a first input token using the distances and the abstract tag transition probabilities; and 
 recognize a first named entity in the input tokens, wherein the first named entity corresponds to the first tag. 
 
 
     
     
       9. The system of  claim 8 , wherein the at least one server computer is configured to compute the input token embeddings using a value of the classifier prior to one or more output layers. 
     
     
       10. The system of  claim 8 , wherein the at least one server computer is configured to assign the first tag from the second tag set to the first input token by computing nearest neighbor scores indicating a match between the first input token and each tag in the second tag set. 
     
     
       11. The system of  claim 10 , wherein the nearest neighbor scores for the first input token are computed using distances between a first token embedding corresponding to the first input token and a nearest support token embedding from the support token embeddings for each tag of the second tag set. 
     
     
       12. The system of  claim 10 , wherein the at least one server computer is configured to assign the first tag from the second tag set to the first input token by processing the nearest neighbor scores with a Viterbi decoder. 
     
     
       13. The system of  claim 8 , wherein the classifier comprises a long short-term memory neural network or a bidirectional encoder representations from transformers neural network. 
     
     
       14. One or more non-transitory, computer-readable media comprising computer-executable instructions that, when executed, cause at least one processor to perform actions comprising:
 obtaining a classifier for a first named entity recognition task corresponding to a first tag set, wherein the classifier processes tokens to compute score vectors for the tokens; 
 obtaining support tokens corresponding to a second named entity recognition task, wherein the support tokens are labelled with a tag from a second tag set, and wherein the second tag set comprises a first tag that is not in the first tag set; 
 obtaining support token embeddings, wherein the support token embeddings were computed by processing the support tokens with the classifier for the first named entity recognition task; 
 obtaining abstract tag transition probabilities, wherein the abstract tag transition probabilities comprise (i) a transition between a tag that does not correspond to any named entity to a tag that corresponds to any named entity, (ii) a transition between a tag that corresponds to any named entity to a tag that does not correspond to any named entity, and (iii) a transition between a tag that corresponds to any named entity to a tag that corresponds to a different named entity; 
 receiving input tokens for processing with the second named entity recognition task; 
 computing input token embeddings by processing the input tokens with the classifier for the first named entity recognition task; 
 computing distances between the input token embeddings and the support token embeddings; 
 assigning the first tag from the second tag set to a first input token using the distances and the abstract tag transition probabilities; and 
 recognizing a first named entity in the input tokens, wherein the first named entity corresponds to the first tag. 
 
     
     
       15. The one or more non-transitory, computer-readable media of  claim 14 , wherein assigning the first tag from the second tag set to a first input token comprises computing, for the first input token, nearest neighbor scores indicating a match between the first input token and each tag in the second tag set. 
     
     
       16. The one or more non-transitory, computer-readable media of  claim 14 , wherein the first named entity corresponds to a person, profession, location, organization, company, number, percentage, date, time, monetary value, phone number, email, zip code, address, product, service, color, medical code, disease, diagnosis, doctor, patient, or customer. 
     
     
       17. The one or more non-transitory, computer-readable media of  claim 14 , wherein the input tokens are word-piece encodings.