Why A/B Split Email Testing Is Invaluable

If you are working as a Project Manager in the interactive space, it is likely that you have (or will) work on an email communication program as part of an overall client strategy. As consumer in-boxes become more and more cluttered with brand messages, it's important that clients learn to compete in this saturated arena. This entry will describe the concept of A/B split testing, and why this very simple tactic can produce big results at low costs.

What is it?: A/B split email testing involves deploying multiple versions of a single email message. In other words, a sample of the total deployment list is split it into two (or more) lists so that each recipient group receives a slightly different message. The variation can be achieved through different subject lines, alternative layouts, varying calls to action, and so on. It is important to isolate the change and link improved results to the variation, so it's best to change only one element at a time. Once the email messages are deployed, results from each variation are measured (fairly quickly, so that the message will remain timely) and the single best performing message is redeployed the remainder of the list. The concept is simple - let's see what generates the highest response rate and send that to the majority of the list to boost results.

Because this approach to A/B testing yields results on a per-message basis, this testing style must be executed with each deployment, if improved results are to be gained each time. The client may garner general insight that can guide future email tactics, but consumer behaviour is not always predictable, and numerous factors are at play with each message sent. It is not enough to conduct a single A/B test and assume learnings can be carried forward without additional testing. This methodology is low-fidelity (simple), and is intended to be a tactic used with each deployment.

Why our clients will appreciate it: A/B testing can represent a very cost-effective approach to generating improved response rates. Put simply, A/B testing has a high ROI (return on investment). It does not require creating two completely different email messages - it only requires making slight variations to a single message, which keeps work effort, time to produce, and overall costs low. More importantly, test results can help debunk myths that will further educate internal and client teams regarding consumer behaviour. Test results may be surprising.

As a Project Manager, you should be pushing for continuous improvement so that client initiatives are successful. Not only will program success lead to an overall pleasant client experience, it will help establish your organization as credible and capable subject matter experts. If you have not already done so, suggest A/B testing as a refinement tactic that can yield immediate results with low investment. A/B testing is a simple step towards gathering consumer insight and research, which are key pillars in achieving continued success in a rapidly shifting medium.

The Value of a Website Prototype

Project Managers are often faced with tremendous pressure to deliver complex initiatives in short periods of time. We all know what it's like to stare at a critical path, having exhausted every option to cut time from the schedule. As a Project Manager, you need to be able to think outside the box and develop new tactics to meet these challenges, even when it seems impossible to do so. In certain cases, a website prototype may be just the answer you're looking for. Let's explore what a website prototype is, and how it can be beneficial to a Project Manager.

What is a website prototype?: In simple terms, a website prototype is a pared down version of what the final product will be. A prototype should include actual creative, copy and functionality, in a much smaller scale than the complete solution. For example, a prototype may be ten pages of a 100 page website. The size and complexity of a prototype can be dictated by time allowances and the specifications of a particular project. As we explore how a prototype is useful, we will touch on factors that will help you determine how large a prototype should be.

Why develop a website prototype?: Developing a website prototype has many benefits - particularly when timelines are tight and the project is complex. A prototype acts as a representation of the final product, which can be used to obtain client feedback at an early stage of the project. Developing a few pages of the full site will give the client an opportunity to comment on creative, content, functionality and overall user experience before the majority of the budget has been spent. Once an entire website has been built, implementing site-wide changes can represent a more significant amount of work.

Prototypes also present an opportunity for team members to work more collaboratively on the overall solution. This will generally lead to a more thoughtful solution.

Most importantly, a website prototype may reveal flaws in usability that can be corrected during the remainder of the production cycle.

When developing a prototype, consider including unique pages, so that any anomalies can be dealt with during the prototype review. When the client approves the prototype, they should be approving look and feel, tone of content, user flow and any functionality that was included. This will allow the team to forge ahead knowing they have received sign-off on significant elements of the build.

From a project management perspective, developing a prototype can streamline production schedules, engage the client earlier on in the project life cycle, and foster teamwork internally. Consider developing a prototype for your next client engagement.

I invite all readers to post comments on their own experiences with website prototypes on my blog.