Automated Multivariate Testing Technique for Optimized Customer Outcome

Abstract

Novel tools and techniques to optimize, from a merchant's perspective, the outcomes of interactions between the merchant and its customers (whether existing, new, or potential customers). In an aspect, certain embodiments allow for multiple variations of options (multivariate or A/B testing) and outcomes which can be statistically applied and then measured and analyzed for best economic impact or preferred outcome.

What is claimed is: 
     
         1 . A method of optimizing outcomes of interactions with customers by optimizing interactive communications systems, the method comprising:
 automatically repeating iteratively for different customers of a plurality of customers, the operations of:
 automatically selecting, by an interactive communications system, one of a plurality of prompt groups for each of the plurality of customers; 
 automatically presenting, by the interactive communications system, a series of prompts to each customer, in accordance with the selected prompt group, each of the prompts providing an offer to the customer, each series of prompts being ordered according to a different predefined path; 
 receiving, from each customer at the interactive communications system, an acceptance of an offer corresponding to one of the prompts; 
 automatically evaluating, by the interactive communications system, the prompt group, based at least in part on the acceptance of the offer, to determine an effectiveness of the series of prompts in producing a preferred outcome that has been determined to be optimal for the merchant, the preferred outcome comprising at least one of an outcome in which a customer keeps the purchased product, an outcome in which an amount of any refund provided to a customer is minimized, an outcome in which satisfaction of existing customers is maximized, an outcome in which one or more new products are sold to a customer, an outcome in which a partial refund is provided when a customer keeps a product intended for return, an outcome in which one or more additional products are sold to a customer regardless of whether the customer returns previously purchased product, an outcome in which a customer accepts a particular offer, an outcome in which one or more of length, frequency, or communication channel of customer interactions is constrained, an outcome in which a series of prompts most quickly results in customer acceptance of an offer, an outcome in which a customer is encouraged to end telephone interaction and resume interaction via a website, or an outcome in which financial loss for the merchant is minimized; and 
 automatically modifying, by the interactive communications system, the prompt group, based at least in part on an evaluation of the prompt group; 
   automatically comparing, by the interactive communications system, a performance of one split test prompt group with the performance of each of the other of the plurality of split test prompt groups to optimize performance of the interactive communications system in interacting with customers.   
     
     
         2 . The method of  claim 1 , further comprising:
 providing, by an interactive communications system, a customer interface for interacting with customers of a merchant;   receiving, at the customer interface, a contact from a customer of the merchant, the customer being one of the plurality of customers;   identifying the customer at the interactive communications system;   determining, by the interactive communications system, a status group of the customer, wherein the status group comprises a plurality of customers, each of the plurality of customers in the status group having similar characteristics with each other customer in the status group;   automatically selecting, by the interactive communications system, a prompt group for the customer, based at least in part on the status group of the customer;   automatically presenting, by the interactive communications system using the customer interface, a series of prompts to the customer in accordance with the selected prompt group, each of the prompts providing an offer to the customer, the series of prompts being ordered according to a predefined path, wherein the series of prompts comprises varying prompts, and wherein the series of prompts associated with a first prompt group differs from the series of prompts associated with a second prompt group by at least one of order, prompt content, or style of prompt;   receiving, at the customer interface, an acceptance of an offer corresponding to one of the prompts;   performing, by the interactive communications system, a fulfillment operation, based on the acceptance of the offer;   automatically evaluating, by the interactive communications system, the prompt group, based at least in part on the acceptance of the offer, to determine an effectiveness of the series of prompts in producing a preferred outcome that has been determined to be optimal for the merchant;   automatically modifying, by the interactive communications system, the prompt group, based at least in part on an evaluation of the prompt group, wherein the prompt group is one of a plurality of split test prompt groups and the customer is one of the plurality of customers.   
     
     
         3 . The method of  claim 1 , wherein a first prompt group comprises a first prompt in a first series of prompts and a second prompt group comprises a second prompt in a second series of prompts, and wherein the first prompt and the second prompt provide the same offer to the customer, and the first prompt is ordered differently in the first series of prompts compared with an order of the second prompt in the second series of prompts. 
     
     
         4 . The method of  claim 1 , wherein the interactive communications system evaluates the prompt group automatically without user input. 
     
     
         5 . The method of  claim 1 , wherein the customer is an existing customer of the merchant, a new customer of the merchant, or a potential customer of the merchant. 
     
     
         6 . The method of  claim 1 , wherein each of the plurality of split test prompt groups includes the same series of prompts, and wherein the series of prompts is ordered differently in each prompt group. 
     
     
         7 . The method of  claim 1 , wherein each of the plurality of split test prompt groups includes at least one prompt different from at least one other of the plurality of split test prompt groups. 
     
     
         8 . The method of  claim 1 , wherein the first prompt delivers the offer to the customer in a first style and the second prompt delivers the offer to the customer in a second style. 
     
     
         9 . The method of  claim 8 , wherein the first style employs different verbiage from the second style. 
     
     
         10 . The method of  claim 8 , wherein the first style employs a different voice from the second style. 
     
     
         11 . The method of  claim 8 , wherein the first style employs a different intonation from the second style. 
     
     
         12 . The method of  claim 1 , wherein modifying the prompt group comprises selecting one of the plurality of prompt groups as an optimal prompt group for the status group of which the customer is a member. 
     
     
         13 . The method of  claim 1 , further comprising:
 determining, by the interactive communications system, the preferred outcome for the interaction with the customer.   
     
     
         14 . The method of  claim 13 , wherein the one or more characteristics of the customer include one or more characteristics selected from the group consisting of: a type of the customer, a source of the customer, an age of an account of the customer, a status of an account of the customer, a purchase history of the customer, and demographic information about the customer. 
     
     
         15 . The method of  claim 13 , wherein determining the preferred outcome further comprises determining the preferred outcome based at least in part on predefined preferences of the merchant. 
     
     
         16 . The method of  claim 1 , wherein performing a fulfillment operation comprises:
 interfacing with a computer system of the merchant; and   communicating the nature of the accepted offer to the computer system of the merchant.   
     
     
         17 . The method of  claim 1 , wherein at least one of the prompts requires the customer to decline or accept a particular offer. 
     
     
         18 . The method of  claim 1 , wherein at least one of the prompts requires the customer to select from among a plurality of offers. 
     
     
         19 . The method of  claim 1 , further comprising:
 maintaining a database of customers of the merchant, the database comprising a plurality of customer records, including a first customer record for the customer of the merchant;   wherein determining a status group of the customer comprises identifying a status of the customer based on analysis of information in the first customer record against business rules specific to the merchant.   
     
     
         20 . The method of  claim 19 , further comprising:
 identifying, in the database, a plurality of customers with similar characteristics; and   assigning different status markers to each of the plurality of customers.   
     
     
         21 . The method of  claim 1 , wherein the customer interface is one of an interactive voice response (“IVR”) interface in which the prompts are audible prompts, a web interface, an electronic mail interface, an electronic chat interface, or a short message service (“SMS”) interface. 
     
     
         22 . The method of  claim 1 , wherein the customer interface is one of a plurality of customer interfaces, the plurality of customer interfaces comprising an interactive voice response (“IVR”) interface, a web interface, and an email or chat or short message service (“SMS”) interface, and wherein the interactive communications system presents the same series of prompts in each of the customer interfaces. 
     
     
         23 . The method of  claim 1 , further comprising:
 automatically analyzing, by the interactive communications system, outcomes of a plurality of customer interactions employing a plurality of prompt groups; and   automatically generating, by the interactive communications system, a report comprising statistical comparisons of an effectiveness of each of the plurality of prompt groups at obtaining an outcome that has been determined to be optimal for the merchant.   
     
     
         24 . An interactive communications device comprising at least one of an interactive voice response (“IVR”) system, an interactive electronic mail (“email”) communications system, an interactive web-based communications system, an interactive electronic chat communications system, or an interactive short message service (“SMS”) communications system, the interactive communications device comprising:
 at least one processor; 
 a non-transitory computer readable medium having encoded thereon a set of instructions executable by the at least one processor to cause the interactive communications device to:
 automatically repeat iteratively for different customers of a plurality of customers, the operations of:
 automatically selecting one of a plurality of prompt groups for each of the plurality of customers; 
 automatically presenting a series of prompts to each customer, in accordance with the selected prompt group, each of the prompts providing an offer to the customer, each series of prompts being ordered according to a different predefined path; 
 receiving, from each customer, an acceptance of an offer corresponding to one of the prompts; 
 automatically evaluating the prompt group, based at least in part on the acceptance of the offer, to determine an effectiveness of the series of prompts in producing a preferred outcome that has been determined to be optimal for the merchant, the preferred outcome comprising at least one of an outcome in which a customer keeps the purchased product, an outcome in which an amount of any refund provided to a customer is minimized, an outcome in which satisfaction of existing customers is maximized, an outcome in which one or more new products are sold to a customer, an outcome in which a partial refund is provided when a customer keeps a product intended for return, an outcome in which one or more additional products are sold to a customer regardless of whether the customer returns previously purchased product, an outcome in which a customer accepts a particular offer, an outcome in which one or more of length, frequency, or communication channel of customer interactions is constrained, an outcome in which a series of prompts most quickly results in customer acceptance of an offer, an outcome in which a customer is encouraged to end telephone interaction and resume interaction via a website, or an outcome in which financial loss for the merchant is minimized; and 
 
 
 automatically modifying the prompt group, based at least in part on an evaluation of the prompt group; 
 automatically comparing a performance of one split test prompt group with the performance of each of the other of the plurality of split test prompt groups to optimize performance of the interactive communications system in interacting with customers. 
 
     
     
         25 . An interactive communications system comprising at least one of an interactive voice response (“IVR”) system, an interactive electronic mail (“email”) communications system, an interactive web-based communications system, an interactive electronic chat communications system, or an interactive short message service (“SMS”) communications system, the interactive communications system comprising:
 one or more processors; 
 a customer interface, in communication with the one or more processors, for interfacing with existing customers of a merchant; and 
 a computer readable medium in communication with the one or more processors, the computer readable medium having encoded thereon a set of instructions executable by the one or more processors to cause the interactive communications system to:
 automatically repeat iteratively for different customers of a plurality of customers, the operations of:
 automatically selecting one of a plurality of prompt groups for each of the plurality of customers; 
 automatically presenting a series of prompts to each customer, in accordance with the selected prompt group, each of the prompts providing an offer to the customer, each series of prompts being ordered according to a different predefined path; 
 receiving, from each customer, an acceptance of an offer corresponding to one of the prompts; 
 automatically evaluating the prompt group, based at least in part on the acceptance of the offer, to determine an effectiveness of the series of prompts in producing a preferred outcome that has been determined to be optimal for the merchant, the preferred outcome comprising at least one of an outcome in which a customer keeps the purchased product, an outcome in which an amount of any refund provided to a customer is minimized, an outcome in which satisfaction of existing customers is maximized, an outcome in which one or more new products are sold to a customer, an outcome in which a partial refund is provided when a customer keeps a product intended for return, an outcome in which one or more additional products are sold to a customer regardless of whether the customer returns previously purchased product, an outcome in which a customer accepts a particular offer, an outcome in which one or more of length, frequency, or communication channel of customer interactions is constrained, an outcome in which a series of prompts most quickly results in customer acceptance of an offer, an outcome in which a customer is encouraged to end telephone interaction and resume interaction via a website, or an outcome in which financial loss for the merchant is minimized; and 
 automatically modifying the prompt group, based at least in part on an evaluation of the prompt group; 
 
 automatically comparing a performance of one split test prompt group with the performance of each of the other of the plurality of split test prompt groups to optimize performance of the interactive communications system in interacting with customers.