Processing event data and/or tabular data for input to one or more machine learning models

Abstract

Aspects described herein may relate to techniques and/or methods that process certain forms of event data and/or tabular data for input to one or more machine learning models, such as a large language model. Additional aspects may relate to using the output of a large language model as part of a process for detecting fraud based on the event data and/or tabular data. In some variations, the event data and/or tabular data may be processed into data tokens, embeddings, or other forms of data suitable for use as input to a large language model.

We claim: 
     
         1 . A method comprising:
 receiving, by one or more computing devices, first event data that includes a plurality of data values for a first event;   for a first data value of the plurality of data values, determining, by the one or more computing devices, a first alphanumeric representation based on a frequency of occurrence of the first data value;   for a second data value of the plurality of data values, determining, by the one or more computing devices, a second alphanumeric representation based on a comparison of the second data value to a threshold value;   for a third data value of the plurality of data values, determining, by the one or more computing devices, a third alphanumeric representation based on the third data value;   determining, by the one or more computing devices, a first sequential data token for the first event that includes the first alphanumeric representation, the second alphanumeric representation, and the third alphanumeric representation in a sequential order;   providing, by the one or more computing devices, the first sequential data token as input to a first language model that outputs generative text;   based on the first sequential data token, receiving, by the one or more computing devices, first generative text from the first language model; and   sending, by the one or more computing devices, the first generative text for use as input to one or more machine-learning models associated with a classification task for the first event data.   
     
     
         2 . The method of  claim 1 , further comprising:
 determining a plurality of sequential data tokens based on the first event data, wherein the first sequential data token is one of the plurality of sequential data tokens;   providing the plurality of sequential data tokens as input to a plurality of language models, wherein the first language model is one of the plurality of language models, and wherein providing the first sequential data token is performed as part of providing the plurality of sequential data tokens as input to the plurality of language models;   based on the plurality of sequential data tokens, receiving generative text from each of the plurality of language models, wherein receiving the first generative text is performed as part of receiving generative text from each of the plurality of language models; and   determining to use the first generative text for the classification task based on providing the generative text received from each of the plurality of language models as input to a machine-learning model that is configured to provide an indication of which language model to use for the classification task;   wherein sending the first generative text for use as input to the one or more machine-learning models associated with the classification task is performed based on determining to use the first generative text for the classification task.   
     
     
         3 . The method of  claim 1  further comprising:
 determining, by the one or more computing devices, a second sequential data token for the first event based on inserting the first data value and the second data value into a natural language template for the first event data; 
 providing, by the one or more computing devices, the second sequential data token as input to a second language model that outputs generative text; and 
 based on the second sequential data token, receiving, by the one or more computing devices, second generative text from the second language model. 
 
     
     
         4 . The method of  claim 1  further comprising:
 determining, by the one or more computing devices, a second sequential data token by applying an index lookup to the first data value to form an indexed version of the first data value, applying an embedding lookup to the indexed version of the first data value to form a first embedding, applying a linear projection to the second data value to form a second embedding, and by combining the first embedding and the second embedding; 
 providing, by the one or more computing devices, the second sequential data token as input to a second language model that outputs generative text; and 
 based on the second sequential data token, receiving, by the one or more computing devices, second generative text from the second language model. 
 
     
     
         5 . The method of  claim 1  further comprising:
 determining, by the one or more computing devices, a second sequential data token for the first event based on an optimization process that selects a subset of data values to include as part of the second sequential data token; 
 providing, by the one or more computing devices, the second sequential data token as input to a second language model that outputs generative text; and 
 based on the second sequential data token, receiving, by the one or more computing devices, second generative text from the second language model. 
 
     
     
         6 . The method of  claim 5 , wherein the optimization process includes performing a Bayesian optimization loop that optimizes based a target variable for the second sequential data token. 
     
     
         7 . The method of  claim 1 , wherein the first event data includes a credit card transaction, the first data value indicates a merchant identifier, the second data value indicates an amount of the credit card transaction, and the third data value indicates whether a credit card was physically present for the credit card transaction;
 wherein the threshold value is configured based on a selected level of granularity; and   wherein the second alphanumeric representation indicates the selected level of granularity and indicates that the amount of the credit card transaction is within a range of prices.   
     
     
         8 . The method of  claim 1 , wherein the first event data includes an account transaction, the first data value indicates a type of account, the second data value indicates an amount for the account transaction, and the third data value indicates whether the transaction was automatic or manual. 
     
     
         9 . The method of  claim 1 , wherein the first event data includes click stream data generated based on one or more user interactions with a user interface, the first data value indicates an identifier for an event initiated based on the one or more user interactions, the second data value indicates a time stamp for the event, and the third data value indicates a type of platform associated with the event. 
     
     
         10 . The method of  claim 1 , wherein the classification task is associated with detecting fraud associated with an account. 
     
     
         11 . An apparatus comprising:
 one or more processors; and   memory storing executable instructions that, when executed by the one or more processors, cause the apparatus to:
 receive first event data that includes a plurality of data values for a first event; 
 for a first data value of the plurality of data values, determine a first alphanumeric representation based on a frequency of occurrence of the first data value; 
 for a second data value of the plurality of data values, determine a second alphanumeric representation based on a comparison of the second data value to a threshold value; 
 for a third data value of the plurality of data values, determine a third alphanumeric representation based on the third data value; 
 determine a first sequential data token for the first event that includes the first alphanumeric representation, the second alphanumeric representation, and the third alphanumeric representation in a sequential order; 
 provide the first sequential data token as input to a first language model that outputs generative text; 
 based on the first sequential data token, receive first generative text from the first language model; and 
 send the first generative text for use as input to one or more machine-learning models associated with a classification task for the first event data. 
   
     
     
         12 . The apparatus of  claim 11 , wherein the executable instructions, when executed by the one or more processors, cause the apparatus to:
 determine a plurality of sequential data tokens based on the first event data, wherein the first sequential data token is one of the plurality of sequential data tokens;   provide the plurality of sequential data tokens as input to a plurality of language models, wherein the first language model is one of the plurality of language models, and wherein the executable instructions, when executed by the one or more processors, cause the apparatus to provide the first sequential data token as part of the plurality of sequential data tokens that are provided as input to the plurality of language models;   based on the plurality of sequential data tokens, receive generative text from each of the plurality of language models, wherein the executable instructions, when executed by the one or more processors, cause the apparatus to receive the first generative text as part of the generative text received from each of the plurality of language models; and   determine to use the first generative text for the classification task based on providing the generative text received from each of the plurality of language models as input to a machine-learning model that is configured to provide an indication of which language model to use for the classification task;   wherein the executable instructions that, when executed by the one or more processors, cause the apparatus to send the first generative text for use as input to the one or more machine-learning models associated with the classification task based on determining to use the first generative text for the classification task.   
     
     
         13 . The apparatus of  claim 11 , wherein the executable instructions, when executed by the one or more processors, cause the apparatus to:
 determine a second sequential data token for the first event based on inserting the first data value and the second data value into a natural language template for the first event data;   provide the second sequential data token as input to a second language model that outputs generative text; and   based on the second sequential data token, receive second generative text from the second language model.   
     
     
         14 . The apparatus of  claim 11 , wherein the executable instructions, when executed by the one or more processors, cause the apparatus to:
 determine a second sequential data token by applying an index lookup to the first data value to form an indexed version of the first data value, applying an embedding lookup to the indexed version of the first data value to form a first embedding, applying a linear projection to the second data value to form a second embedding, and by combining the first embedding and the second embedding;   provide the second sequential data token as input to a second language model that outputs generative text; and   based on the second sequential data token, receive second generative text from the second language model.   
     
     
         15 . The apparatus of  claim 11 , wherein the executable instructions, when executed by the one or more processors, cause the apparatus to:
 determine a second sequential data token for the first event based on an optimization process that selects a subset of data values to include as part of the second sequential data token;   provide the second sequential data token as input to a second language model that outputs generative text; and   based on the second sequential data token, receive second generative text from the second language model.   
     
     
         16 . The apparatus of  claim 11 , wherein the first event data includes a credit card transaction, the first data value indicates a merchant identifier, the second data value indicates an amount of the credit card transaction, and the third data value indicates whether a credit card was physically present for the credit card transaction;
 wherein the threshold value is configured based on a selected level of granularity; and   wherein the second alphanumeric representation indicates the selected level of granularity and indicates that the amount of the credit card transaction is within a range of prices.   
     
     
         17 . One or more non-transitory computer-readable media storing executable instructions that, when executed, cause one or more computing devices to:
 receive first event data that includes a plurality of data values for a first event;   for a first data value of the plurality of data values, determine a first alphanumeric representation based on a frequency of occurrence of the first data value;   for a second data value of the plurality of data values, determine a second alphanumeric representation based on a comparison of the second data value to a threshold value;   for a third data value of the plurality of data values, determine a third alphanumeric representation based on the third data value;   determine a first sequential data token for the first event that includes the first alphanumeric representation, the second alphanumeric representation, and the third alphanumeric representation in a sequential order;   provide the first sequential data token as input to a first language model that outputs generative text;   based on the first sequential data token, receive first generative text from the first language model; and   send the first generative text for use as input to one or more machine-learning models associated with a classification task for the first event data.   
     
     
         18 . The one or more non-transitory computer-readable media of  claim 17 , wherein the executable instructions, when executed, cause the one or more computing devices to:
 determine a plurality of sequential data tokens based on the first event data, wherein the first sequential data token is one of the plurality of sequential data tokens;   provide the plurality of sequential data tokens as input to a plurality of language models, wherein the first language model is one of the plurality of language models, and wherein the executable instructions, when executed, cause the one or more computing devices to provide the first sequential data token as part of the plurality of sequential data tokens that are provided as input to the plurality of language models;   based on the plurality of sequential data tokens, receive generative text from each of the plurality of language models, wherein the executable instructions, when executed, cause the one or more computing devices to receive the first generative text as part of the generative text received from each of the plurality of language models; and   determine to use the first generative text for the classification task based on providing the generative text received from each of the plurality of language models as input to a machine-learning model that is configured to provide an indication of which language model to use for the classification task;   wherein the executable instructions that, when executed, cause the one or more computing devices to send the first generative text for use as input to the one or more machine-learning models associated with the classification task based on determining to use the first generative text for the classification task.   
     
     
         19 . The one or more non-transitory computer-readable media of  claim 17 , wherein the executable instructions, when executed, cause the one or more computing devices to:
 determine a second sequential data token for the first event based on inserting the first data value and the second data value into a natural language template for the first event data;   provide the second sequential data token as input to a second language model that outputs generative text; and   based on the second sequential data token, receive second generative text from the second language model.   
     
     
         20 . The one or more non-transitory computer-readable media of  claim 17 , wherein the executable instructions, when executed, cause the one or more computing devices to:
 determine a second sequential data token by applying an index lookup to the first data value to form an indexed version of the first data value, applying an embedding lookup to the indexed version of the first data value to form a first embedding, applying a linear projection to the second data value to form a second embedding, and by combining the first embedding and the second embedding;   provide the second sequential data token as input to a second language model that outputs generative text; and   based on the second sequential data token, receive second generative text from the second language model.