Adaptive gateway switching system

Abstract

Systems and methods disclosed herein relate to reducing the losses associated with processing payments using credit card payment processing gateways. Payment processing gateways have different deposit success rates due to the gateways' varying arrangements with acquiring banks and the ability to process certain types of payments. A payment request related to a user may be received, wherein the request specifies a particular type of credit card, a transaction currency, and an amount. A payment application then routes, via a payment agent, the payment request to an adaptive gateway switching system (AGSS) in an attempt to complete a payment transaction. If in learning mode, the AGSS may randomly choose a payment processing gateway. Or the AGSS may choose the gateway based upon a learned gateway status based upon results learned through reinforcement learning. Upon receiving the result of a transaction attempt from the gateway, the agent reports the result to the AGSS. The result is used to update a reward function to inform future attempts to process payments. If the transaction attempt was unsuccessful, the AGSS provides an alternative gateway option for the agent to contact and attempt the transaction. This may continue until one of the following occurs: success; all gateways have been attempted; a predetermined number of gateways has been attempted; a predetermined penalty threshold has been exceeded; or another relevant criterion has been satisfied.

What is claimed is: 
     
         1 . A method, comprising:
 receiving with a web server a transaction request from a remote user computing device via hypertext transfer protocol,   wherein the transaction request includes a specification of a currency, an amount, and account information,   wherein the account information includes at least a card type, an issuing bank country, and an account identifier; and   performing the following steps within a 30 second period without further human intervention:   storing the currency, the amount and the account information in a physical machine memory;   sending to a remote payment processing gateway computing device a gateway processing request based on the transaction request via a gateway payment protocol,   wherein the gateway processing request includes specification of the currency, the amount, and at least a portion of the account information including the account identifier,   wherein the payment processing gateway is chosen from a plurality of payment processing gateways based upon a policy, and an identification of the payment processing gateway and an internet address of the remote payment processing gateway computing device is retrieved from a first memory device,   wherein the identification of the payment processing gateway is based upon the policy,   wherein the policy defines a processing gateway to choose based upon at least the specification of the currency and a portion of the account information that includes at least the card type and the issuing bank country;   receiving with a payment server a result of the gateway processing request from the remote payment processing gateway computing device via the gateway payment protocol; and   after receiving the result:   updating the policy based upon the result,   wherein updating the policy includes computing a real number reward based upon the result;   storing the updated portion of the policy in a memory device, which may be the first memory device or a second memory device; and   if the result indicates that the gateway was unable to process the gateway processing request,   sending to a second remote payment processing gateway computing device a second gateway processing request based on the transaction request via a gateway payment protocol of the second gateway,   wherein the second gateway processing request includes specification of the currency, the amount, and at least a portion of the account information including the account identifier,   wherein the second payment processing gateway is chosen from the plurality of payment processing gateways based upon the updated policy, and an identification of the second payment processing gateway and an internet address of the second remote payment processing gateway computing device is retrieved from the first memory device,   wherein the identification of the second payment processing gateway is based upon the updated policy.   
     
     
         2 . The method of  claim 1 , further comprising:
 receiving with the payment server a result of the second gateway processing request from the second payment processing gateway via the gateway payment protocol of the second payment processing gateway;
 after receiving the result of the second request: 
 updating the policy based upon the result of the second request, 
 wherein updating the policy includes computing a real number reward based upon the result of the second request; 
 storing the updated portion of the policy in the memory device; 
 if the result of the second request indicates that the second gateway was unable to process the second gateway processing request, 
 continuing to attempt to process the transaction request via successive gateway processing requests by repeating the following steps until a termination condition is reached, 
 wherein the termination condition is chosen from the following conditions: (1) a result of a gateway processing request indicates success, (2) at least one gateway processing request has been sent to each of the plurality of payment processing gateways, and (3) the policy indicates that no further gateway processing requests should be sent; and 
 and the steps to repeat comprise: 
 sending to another payment processing gateway another gateway processing request based on the transaction request via another gateway payment protocol, 
 wherein the other gateway processing request includes specification of the currency, the amount, and at least a portion of the account information including the account identifier, 
 wherein the other payment processing gateway is chosen from a plurality of payment processing gateways based upon the policy, and an identification of the other payment processing gateway is retrieved from the first memory device, 
 wherein the identification of the other payment processing gateway is based upon the policy, 
 wherein the policy defines the other processing gateway to choose based upon at least the specification of the currency and a portion of the account information that includes at least the card type and the issuing bank country; 
 receiving with the payment server another result of the other gateway processing request from the other payment processing gateway via the other gateway payment protocol; and 
 after receiving the other result: 
 updating the policy based upon the other result, 
 wherein updating the policy includes computing a real number reward based upon the other result; 
 storing the updated portion of the policy in the memory device. 
   
     
     
         3 . The method of  claim 1 , further comprising:
 repeating the steps of  claim 1  for at least 1000 transaction requests within a 24 hour period.   
     
     
         4 . The method of  claim 2 , further comprising:
 repeating the steps of  claim 2  for at least 1000 transaction requests within a 24 hour period.   
     
     
         5 . The method of  claim 1 , further comprising:
 repeating the steps of  claim 1  for at least 1000 transaction requests within an 8 hour period.   
     
     
         6 . The method of  claim 2 , further comprising:
 repeating the steps of  claim 2  for at least 1000 transaction requests within an 8 hour period.   
     
     
         7 . The method of  claim 2 , wherein,
 the transaction request is a request to make a payment using a credit card.   
     
     
         8 . The method of  claim 2 , wherein,
 each result is chosen from the set consisting of:   (a) success;   (b) failure; and   (c) inability to complete the gateway processing request due to expiration of a predetermined time out period for the gateway processing request.   
     
     
         9 . The method of  claim 2 , wherein,
 the steps performed after receiving the result of the second request are performed within a 30 second period without further human intervention.   
     
     
         10 . The method of  claim 2 , wherein,
 the policy provides a mapping of a plurality of states to a plurality of potential actions, and each of the plurality of states is mapped to one of the plurality of potential actions;   wherein the plurality of potential actions comprises identifications of payment processing gateways to which processing requests may be directed;   wherein each state comprises a tuple including a card type, a transaction currency, a country of a card issuer, a result of the previous processing request, and an ordered list of zero or more payment processing gateways to which previous processing requests have been sent for the relevant card type, currency, and card issuer country.   
     
     
         11 . A method for processing a transaction request, comprising:
 (a) providing a user interface application to a user for installation on a computing device of the user, wherein the application includes functionality for creating transaction requests;   (b) receiving with a server a transaction request sent over the Internet from the computing device, the server comprising a microprocessor and a memory that stores information related to the transaction request;   (c) retrieving data related to the transaction request, including a specified currency, a transaction amount, and an account identifier;   (d) providing the data related to the transaction request to a learning agent;   (e) receiving from the learning agent an identification of a payment processing gateway, wherein the identification of the payment processing gateway by the learning agent is based upon a policy and analysis of the data related to the transaction request;   (f) transmitting to the identified payment processing gateway over the Internet a request to complete a transaction based upon at least a portion of the data related to the transaction request;   (g) receiving from the identified payment processing gateway a result of the request to complete a transaction;   (h) providing the result to the learning agent for use in calculating a reward function and updating the policy based upon the result and the calculated reward; and   (i) transmitting a transaction request result over the Internet to the computing device of the user to cause the user interface application to display a notification alerting the user of the transaction request result and to enable additional functionality in the user interface application related to the transaction request result.   
     
     
         12 . The method of  claim 11 , further comprising:
 (j) if the result indicates that the request to complete a transaction failed, considering whether a termination threshold has been met before transmitting the transaction request result, and   if the termination threshold has not been met, repeating steps (e), (f), (g), (h), and (j) before performing step (i).   
     
     
         13 . The method of  claim 11 , wherein the calculated reward comprises a real number. 
     
     
         14 . The method of  claim 13 , wherein calculating a reward function comprises calculating a result of the equation: 
       
         
           
             
               
                 r 
                  
                 
                   ( 
                   
                     s 
                     , 
                     a 
                     , 
                     f 
                   
                   ) 
                 
               
               = 
               
                 
                   δ 
                    
                   
                     ( 
                     f 
                     ) 
                   
                 
                  
                 
                   
                     ( 
                     
                       1 
                       - 
                       
                         ( 
                         
                           
                             Cost 
                             
                               
                                 g 
                                 i 
                               
                                
                               
                                 ( 
                                 
                                   
                                     s 
                                     · 
                                     M 
                                   
                                   , 
                                   
                                     s 
                                     · 
                                     U 
                                   
                                   , 
                                   f 
                                 
                                 ) 
                               
                             
                           
                           
                             
                               max 
                               
                                 g 
                                 i 
                               
                             
                              
                             
                                 
                             
                              
                             
                               
                                 Cost 
                                 
                                   g 
                                   i 
                                 
                               
                                
                               
                                 ( 
                                 
                                   
                                     s 
                                     · 
                                     M 
                                   
                                   , 
                                   
                                     s 
                                     · 
                                     U 
                                   
                                   , 
                                   f 
                                 
                                 ) 
                               
                             
                           
                         
                         ) 
                       
                     
                     ) 
                   
                   . 
                 
               
             
           
         
       
     
     
         15 . The method of  claim 11 , wherein
 the policy defines a set of actions to take given a particular state of an environment,   wherein the environment comprises a plurality of states, each state being mapped to one of the set of actions, and   wherein the plurality of states are defined based upon the results of a plurality of requests to complete transactions.   
     
     
         16 . The method of  claim 12 , wherein,
 each result is chosen from the set consisting of:   (a) success;   (b) failure; and   (c) inability to complete the gateway processing request due to expiration of a predetermined time out period for the gateway processing request.   
     
     
         17 . The method of  claim 12 , wherein,
 the transaction request is a request to make a payment using a credit card.   
     
     
         18 . The method of  claim 11 , further comprising:
 repeating the steps of  claim 11  for at least 1000 transaction requests within a 24 hour period.   
     
     
         19 . The method of  claim 12 , further comprising:
 repeating the steps of  claim 12  for at least 1000 transaction requests within a 24 hour period.   
     
     
         20 . The method of  claim 12 , wherein,
 the policy provides a mapping of a plurality of states to a plurality of potential actions, and each of the plurality of states is mapped to one of the plurality of potential actions;   wherein the plurality of potential actions comprises identifications of payment processing gateways to which processing requests may be directed; and   wherein each of the plurality of states comprises a tuple including a card type, a transaction currency, a country of the card issuer, a result of the previous processing request, and an ordered list of zero or more payment processing gateways to which previous processing requests have been sent for the relevant card type, currency, and card issuer country.