Why A/B Split Email Testing Is Invaluable

If you are working as a Project Manager in the interactive space, it is likely that you have (or will) work on an email communication program as part of an overall client strategy. As consumer in-boxes become more and more cluttered with brand messages, it's important that clients learn to compete in this saturated arena. This entry will describe the concept of A/B split testing, and why this very simple tactic can produce big results at low costs.

What is it?: A/B split email testing involves deploying multiple versions of a single email message. In other words, a sample of the total deployment list is split it into two (or more) lists so that each recipient group receives a slightly different message. The variation can be achieved through different subject lines, alternative layouts, varying calls to action, and so on. It is important to isolate the change and link improved results to the variation, so it's best to change only one element at a time. Once the email messages are deployed, results from each variation are measured (fairly quickly, so that the message will remain timely) and the single best performing message is redeployed the remainder of the list. The concept is simple - let's see what generates the highest response rate and send that to the majority of the list to boost results.

Because this approach to A/B testing yields results on a per-message basis, this testing style must be executed with each deployment, if improved results are to be gained each time. The client may garner general insight that can guide future email tactics, but consumer behaviour is not always predictable, and numerous factors are at play with each message sent. It is not enough to conduct a single A/B test and assume learnings can be carried forward without additional testing. This methodology is low-fidelity (simple), and is intended to be a tactic used with each deployment.

Why our clients will appreciate it: A/B testing can represent a very cost-effective approach to generating improved response rates. Put simply, A/B testing has a high ROI (return on investment). It does not require creating two completely different email messages - it only requires making slight variations to a single message, which keeps work effort, time to produce, and overall costs low. More importantly, test results can help debunk myths that will further educate internal and client teams regarding consumer behaviour. Test results may be surprising.

As a Project Manager, you should be pushing for continuous improvement so that client initiatives are successful. Not only will program success lead to an overall pleasant client experience, it will help establish your organization as credible and capable subject matter experts. If you have not already done so, suggest A/B testing as a refinement tactic that can yield immediate results with low investment. A/B testing is a simple step towards gathering consumer insight and research, which are key pillars in achieving continued success in a rapidly shifting medium.