Systems and methods for electronic messaging testing optimization in prospect electronic messages series

Abstract

Disclosed are methods, systems, and non-transitory computer-readable medium for optimizing user experience with respect to email journeys. For example, a method may include determining a first development subset, a first quality assurance subset, and a first production subset of a first plurality of user identifiers; determining and transmitting a first development set of messages, a first quality assurance set of messages, and a first production set of message; determining a development performance indicator, a quality assurance performance indicator, and a production performance indicator based on one or more messages received in response to the transmitted first development set of messages, first quality assurance set of messages, and first production set of message, respectively.

1 - 20 . (canceled) 
     
     
         21 . A computer-implemented method comprising:
 dynamically allocating, by one or more processors, a first plurality of user identifiers to a first development subset, a first quality assurance subset, and a first production subset, wherein each of the first plurality of user identifiers is associated with a customer;   determining, by the one or more processors, a first development set of email messages at least based on data regarding an electronic transactions platform;   determining, by the one or more processors, a development performance indicator based on one or more messages received in response to transmitting the first development set of email messages to the first development subset of user identifiers;   determining, by the one or more processors, a first quality assurance set of email messages to the first quality assurance subset of user identifiers;   determining, by the one or more processors, a first production set of email messages for the first production subset of user identifiers;   determining, by the one or more processors, a production performance indicator based on one or more messages received in response to transmitting the first production set of email messages to the first production subset of user identifiers; and   determining a second development set of email messages by:
 comparing the development performance indicator to the production performance indicator; and 
 as a result of determining that the development performance indicator is lower than the production performance indicator, determining and assigning a new set of email messages as the second development set of email messages. 
   
     
     
         22 . The computer-implemented method of  claim 21 , wherein the first production subset includes a larger number of user identifiers than the first quality assurance subset, and wherein the first quality assurance subset includes a larger number of user identifiers than the first development subset. 
     
     
         23 . The computer-implemented method of  claim 21 , further comprising:
 determining, by one or more processors, a second development subset, a second quality assurance subset, and a second production subset of a second plurality of user identifiers, wherein each of the second plurality of user identifiers is associated with a customer;   determining, by the one or more processors, a second development set of email messages at least based on the determined development performance indicator and the determined production performance indicator;   transmitting, by the one or more processors, the second development set of email messages to the second development subset of user identifiers;   determining, by the one or more processors, a second quality assurance set of email messages at least based on the determined development performance indicator and the determined production performance indicator;   transmitting, by the one or more processors, the second quality assurance set of email messages to the second quality assurance subset of user identifiers;   determining, by the one or more processors, a second production set of email messages at least based on the determined quality assurance performance indicator and the determined production performance indicator; and   transmitting, by the one or more processors, the second production set of email messages to the second production subset of user identifiers.   
     
     
         24 . The computer-implemented method of  claim 23 , wherein the second production subset includes a larger number of user identifiers than the second quality assurance subset, and wherein the second quality assurance subset includes a larger number of user identifiers than the second development subset. 
     
     
         25 . The method of  claim 21 , further comprising:
 transmitting, by the one or more processors, the first development set of email messages to the first development subset of user identifiers;   receiving, by the one or more processors, the one or more messages in response to the first development set of email messages transmitted to the first development subset of user identifiers;   transmitting, by the one or more processors, the first quality assurance set of email messages to the first quality assurance subset of user identifiers by assigning a previously transmitted development set of email messages as the first quality assurance set of email messages based on a previously determined development performance indicator for the previously transmitted development set of email messages;   receiving, by the one or more processors, one or more messages in response to the first quality assurance set of email messages transmitted to the first quality assurance subset of user identifiers;   determining, by the one or more processors, a quality assurance performance indicator based on the one or more messages received in response to the first quality assurance set of email messages transmitted to the first quality assurance subset of user identifiers;   transmitting, by the one or more processors, the first production set of email messages to the first production subset of user identifiers by assigning a previously transmitted quality assurance set of email messages as the first production set of email messages based on a previously determined quality assurance performance indicator for the previously transmitted quality assurance set of email messages; and   receiving, by the one or more processors, one or more messages in response to the first production set of email messages transmitted to the first production subset of user identifiers.   
     
     
         26 . The method of  claim 23 , wherein determining the second quality assurance set of email messages at least based on the determined development performance indicator and the determined production performance indicator comprises:
 comparing the determined development performance indicator to the determined production performance indicator; and   as a result of determining that the determined development performance indicator exceeds the determined production performance indicator, replacing the first quality assurance set of email messages with the first development set of email messages to form the second quality assurance set of email messages.   
     
     
         27 . The method of  claim 23 , wherein determining the second production set of email messages at least based on the determined quality assurance performance indicator and the determined production performance indicator comprises:
 comparing the determined quality assurance performance indicator to the determined production performance indicator; and   as a result of determining that the determined quality assurance performance indicator exceeds the determined production performance indicator, replacing the first production set of email messages with the first quality assurance set of email messages to form the second production set of email messages.   
     
     
         28 . A computer system comprising:
 a data storage device storing processor-readable instructions; and   a processor configured to execute the instructions to perform a method including:   dynamically allocating a first plurality of user identifiers to a first quality assurance subset, and a first production subset, wherein each of the first plurality of user identifiers is associated with a customer;   determining a first development set of email messages at least based on data regarding an electronic transactions platform;   determining a development performance indicator based on one or more messages received in response to transmitting the first development set of email messages to the first development subset of user identifiers;   determining a first quality assurance set of email messages to the first quality assurance subset of user identifiers;   determining a first production set of email messages to the first production subset of user identifiers;   determining a production performance indicator based on one or more messages received in response to transmitting the first production set of email messages to the first production subset of user identifiers; and   determining a second development set of email messages by:
 comparing the determined development performance indicator to the determined production performance indicator; and 
 as a result of determining that the determined development performance indicator is lower than the determined production performance indicator, determining and assigning a new set of email messages as the second development set of email messages. 
   
     
     
         29 . The computer system of  claim 28 , wherein the first production subset includes a larger number of user identifiers than the first quality assurance subset, and wherein the first quality assurance subset includes a larger number of user identifiers than the first development subset. 
     
     
         30 . The computer system of  claim 28 , wherein the processor is further configured for:
 determining a second development subset, a second quality assurance subset, and a second production subset of a second plurality of user identifiers, wherein each of the second plurality of user identifiers is associated with a customer;   determining a second development set of email messages at least based on the determined development performance indicator and the determined production performance indicator;   transmitting the second development set of email messages to the second development subset of user identifiers;   determining a second quality assurance set of email messages at least based on the determined development performance indicator and the determined production performance indicator;   transmitting the second quality assurance set of email messages to the second quality assurance subset of user identifiers;   determining a second production set of email messages at least based on the determined quality assurance performance indicator and the determined production performance indicator; and   transmitting the second production set of email messages to the second production subset of user identifiers.   
     
     
         31 . The computer system of  claim 30 , wherein the second production subset includes a larger number of user identifiers than the second quality assurance subset, and wherein the second quality assurance subset includes a larger number of user identifiers than the second development subset. 
     
     
         32 . The computer system of  claim 28 , wherein the processor is further configured for:
 transmitting the first development set of email messages to the first development subset of user identifiers;   receiving the one or more messages in response to the first development set of email messages transmitted to the first development subset of user identifiers;   transmitting the first quality assurance set of email messages to the first quality assurance subset of user identifiers by assigning a previously transmitted development set of email messages as the first quality assurance set of email messages based on a previously determined development performance indicator for the previously transmitted development set of email messages;   receiving one or more messages in response to the first quality assurance set of email messages transmitted to the first quality assurance subset of user identifiers;   determining a quality assurance performance indicator based on the one or more messages received in response to the first quality assurance set of email messages transmitted to the first quality assurance subset of user identifiers;   transmitting the first production set of email messages to the first production subset of user identifiers by assigning a previously transmitted quality assurance set of email messages as the first production set of email messages based on a previously determined quality assurance performance indicator for the previously transmitted quality assurance set of email messages; and   receiving one or more messages in response to the first production set of email messages transmitted to the first production subset of user identifiers.   
     
     
         33 . The computer system of  claim 30 , wherein determining the second quality assurance set of email messages at least based on the determined development performance indicator and the determined production performance indicator comprises:
 comparing the determined development performance indicator to the determined production performance indicator; and   as a result of determining that the determined development performance indicator exceeds the determined production performance indicator, replacing the first quality assurance set of email messages with the first development set of email messages to form the second quality assurance set of email messages.   
     
     
         34 . The computer system of  claim 30 , wherein determining the second production set of email messages at least based on the determined quality assurance performance indicator and the determined production performance indicator comprises:
 comparing the determined quality assurance performance indicator to the determined production performance indicator; and   as a result of determining that the determined quality assurance performance indicator exceeds the determined production performance indicator, replacing the first production set of email messages with the first quality assurance set of email messages to form the second production set of email messages.   
     
     
         35 . A non-transitory computer-readable medium containing instructions that, when executed by a processor, cause the processor to perform a method comprising:
 dynamically allocating a first plurality of user identifiers to a first quality assurance subset, and a first production subset, wherein each of the first plurality of user identifiers is associated with a customer;   determining a first development set of email messages at least based on data regarding an electronic transactions platform;   determining a development performance indicator based on one or more messages received in response to transmitting the first development set of email messages to the first development subset of user identifiers;   determining a first quality assurance set of email messages to the first quality assurance subset of user identifiers;   determining a first production set of email messages to the first production subset of user identifiers;   determining a production performance indicator based on one or more messages received in response to transmitting the first production set of email messages to the first production subset of user identifiers; and   determining a second development set of email messages by:
 comparing the determined development performance indicator to the determined production performance indicator; and 
 as a result of determining that the determined development performance indicator is lower than the determined production performance indicator, determining and assigning a new set of email messages as the second development set of email messages. 
   
     
     
         36 . The non-transitory computer-readable medium of  claim 35 , wherein the first production subset includes a larger number of user identifiers than the first quality assurance subset, and wherein the first quality assurance subset includes a larger number of user identifiers than the first development subset. 
     
     
         37 . The non-transitory computer-readable medium of  claim 35 , the method further comprising:
 determining a second development subset, a second quality assurance subset, and a second production subset of a second plurality of user identifiers, wherein each of the second plurality of user identifiers is associated with a customer;   determining a second development set of email messages at least based on the determined development performance indicator and the determined production performance indicator;   transmitting the second development set of email messages to the second development subset of user identifiers;   determining a second quality assurance set of email messages at least based on the determined development performance indicator and the determined production performance indicator;   transmitting the second quality assurance set of email messages to the second quality assurance subset of user identifiers;   determining a second production set of email messages at least based on the determined quality assurance performance indicator and the determined production performance indicator; and   transmitting the second production set of email messages to the second production subset of user identifiers.   
     
     
         38 . The non-transitory computer-readable medium of  claim 35 , the method further comprising:
 transmitting the first development set of email messages to the first development subset of user identifiers;   receiving the one or more messages in response to the first development set of email messages transmitted to the first development subset of user identifiers;   transmitting the first quality assurance set of email messages to the first quality assurance subset of user identifiers by assigning a previously transmitted development set of email messages as the first quality assurance set of email messages based on a previously determined development performance indicator for the previously transmitted development set of email messages;   receiving one or more messages in response to the first quality assurance set of email messages transmitted to the first quality assurance subset of user identifiers;   determining a quality assurance performance indicator based on the one or more messages received in response to the first quality assurance set of email messages transmitted to the first quality assurance subset of user identifiers;   transmitting the first production set of email messages to the first production subset of user identifiers by assigning a previously transmitted quality assurance set of email messages as the first production set of email messages based on a previously determined quality assurance performance indicator for the previously transmitted quality assurance set of email messages; and   receiving one or more messages in response to the first production set of email messages transmitted to the first production subset of user identifiers.   
     
     
         39 . The non-transitory computer-readable medium of  claim 37 , wherein determining the second quality assurance set of email messages at least based on the determined development performance indicator and the determined production performance indicator comprises:
 comparing the determined development performance indicator to the determined production performance indicator; and   as a result of determining that the determined development performance indicator exceeds the determined production performance indicator, replacing the first quality assurance set of email messages with the first development set of email messages to form the second quality assurance set of email messages.   
     
     
         40 . The non-transitory computer-readable medium of  claim 37 , wherein determining the second production set of email messages at least based on the determined quality assurance performance indicator and the determined production performance indicator comprises:
 comparing the determined quality assurance performance indicator to the determined production performance indicator; and   as a result of determining that the determined quality assurance performance indicator exceeds the determined production performance indicator, replacing the first production set of email messages with the first quality assurance set of email messages to form the second production set of email messages.