Methods and systems for using reinforcement learning

Abstract

Methods and Systems for using reinforcement learning to optimize promotions. A promotion can be offered to a customer for a prepaid calling card using a reinforcement learning model with a sensitivity parameter. The reinforcement learning model can estimate a time period during which the customer will purchase the prepaid calling card. The customer's reaction to the promotion can be observed. A reward or a penalty can be collected based on the customer's reaction. The reinforcement learning model can be adapted based on the reward or the penalty to optimize the timing of the promotion by estimating a new time period during which the customer will purchase the prepaid calling card. The reward proxy and/or the penalty proxy can comprise frequency of usage.

The invention claimed is: 
     
       1. A method of using reinforcement learning to optimizing promotions, comprising:
 a computer device comprising a processor configured for: 
 offering a promotion to a customer for a prepaid calling card using a reinforcement learning model with a sensitivity parameter, the reinforcement learning model estimating a time period during which the customer will purchase the prepaid calling card; 
 observing the customer's reaction to the promotion; 
 collecting a reward or a penalty based on the customer's reaction; and 
 adapting the reinforcement learning model based on the reward or the penalty to optimize the timing of the promotion by estimating a new time period during which the customer will purchase the prepaid calling card; 
 wherein the reward proxy and/or the penalty proxy comprises frequency of usage. 
 
     
     
       2. The method of  claim 1 , wherein the reward or penalty comprises a reward proxy or a penalty proxy indicating that the customer is responding to the promotion. 
     
     
       3. The method of  claim 2 , wherein the reward proxy comprises the customer using the prepaid card more often. 
     
     
       4. The method of  claim 2 , wherein the penalty proxy comprises the customer using the prepaid card less often. 
     
     
       5. The method of  claim 1 , wherein the reinforcement learning model utilizes a customer latent embedding procedure. 
     
     
       6. The method of  claim 1 , wherein the reinforcement learning model utilizes a customer similarity metric. 
     
     
       7. The method of  claim 1 , further comprising:
 receiving customer inputs and/or a set of feature weightings; 
 generating pairs of customers, wherein the pairs of customers can be validated by a user; 
 adapting the reinforcement learning model so that validation by the customer increases; and 
 outputting a customer similarity metric. 
 
     
     
       8. The method of  claim 7 , further comprising using the customer similarity metric to cluster customer inputs into discrete states for the reinforcement learning model. 
     
     
       9. The method of  claim 8 , wherein the method further comprises receiving customer inputs and/or a set of feature weightings, and generating pairs of customers, wherein the pairs of customers can be validated by a user. 
     
     
       10. The method of  claim 9 , wherein the method further comprises adapting the reinforcement learning model so that validation by the customer increases. 
     
     
       11. A system of using reinforcement learning to optimizing promotions, comprising:
 a computing device comprising a processor configured for: 
 offering a promotion to a customer for a prepaid calling card using a reinforcement learning model with a sensitivity parameter, the reinforcement learning model estimating a time period during which a customer will purchase the prepaid calling card; 
 observing the customer's reaction to the promotion; 
 collecting a reward or a penalty based on the customer's reaction; and 
 adapting the reinforcement learning model based on the reward or the penalty to optimize the timing of the promotion by estimating a new time period during which the customer will purchase the prepaid calling card; 
 wherein the reward proxy and/or the penalty proxy comprises frequency of usage. 
 
     
     
       12. The system of  claim 11 , wherein the reward or penalty comprises a reward proxy or a penalty proxy indicating that the customer is responding to the promotion. 
     
     
       13. The system of  claim 12 , wherein the reward proxy comprises the customer using the prepaid card more often. 
     
     
       14. The system of  claim 12 , wherein the penalty proxy comprises the customer using the prepaid card less often. 
     
     
       15. The system of  claim 11 , wherein the reinforcement learning model utilizes a customer latent embedding procedure. 
     
     
       16. The system of  claim 11 , wherein the reinforcement learning model utilizes a customer similarity metric. 
     
     
       17. The system of  claim 11 , wherein the processor is further configured for:
 receiving customer inputs and/or a set of feature weightings; 
 generating pairs of customers, wherein the pairs of customers can be validated by a user; 
 adapting the reinforcement learning model so that validation by the customer increases; and 
 outputting a customer similarity metric. 
 
     
     
       18. The method of  claim 17 , wherein the processor is further configured for using the customer similarity metric to cluster customer inputs into discrete states for the reinforcement learning model. 
     
     
       19. The method of  claim 18 , wherein the processor is further configured for receiving customer inputs and/or a set of feature weightings, and generating pairs of customers, wherein the pairs of customers can be validated by a user. 
     
     
       20. The method of  claim 19 , wherein the processor if further configured for adapting the reinforcement learning model so that validation by the customer increases.