What’s the ROI on your email marketing efforts?
Maybe you’re used to answering that question by explaining that, according to your last-click attribution set-up, 60% of people purchase because of email. Or maybe you’d say something like, “Well, according to our attribution model, we have a 1000% ROI on our email marketing.”
That sounds good and all, but what is the real impact that your email marketing has on paying customers?
For example, how many subscribers would have purchased even if you hadn’t sent that cart abandonment email, meaning that, for those subscribers, that email had no impact at all? Conversely, how many subscribers purchased only because they received your 10-message email sequence and wouldn’t have done so otherwise?
If you can answer those questions, then you can finally determine — black and white, plain and simple — how much revenue your email marketing efforts produce, what kind of results you’re pulling for your client(s), and ultimately, how valuable you are as an email marketer.
The answer?
Control groups.
What are email marketing control groups?
Email marketing control groups are a new (and much needed) way for email marketers to test the real impact of their efforts on revenue and percent of purchases.
For a long time, muddied and half-accurate attribution models along with inconsistent A/B tests have held the reins because… well, it’s what most of us email marketers learned how to do.
And while those testing methods and attribution models are helpful, they’re not as revealing as we’d like them to be, especially when it comes to determining an email campaign’s impact on revenue and conversion.
Control groups, on the other hand, allow you to run statistically significant tests that collect meaningful data for your email marketing efforts.
Put simply, a control group test is not sending an email campaign or series of email campaigns to a portion of your email list in order to determine how big of an impact those emails have on your subscriber’s motivation to convert.
Perhaps the biggest application of this type of test is to determine the overall value of your email marketing efforts. By not sending emails to a portion of your list (say, 10%) for a period of time (say, 2 weeks), and by doing everything else as you’ve always done it for the remaining 90% of your list, you can find out once-and-for-all how big of an impact your email campaigns have on purchases and conversion.
You just need to compare results for the control group (those you didn’t send emails to) and the test group to see what percentage of conversions came from each during the experiment — that will reveal how much of an impact your email marketing as a whole is having on revenue.
But you can also use control groups to, more specifically, test the impact of any new set of emails you want to start sending, the effectiveness of triggered emails (abandoned cart, coupon, and other interest-based campaigns), and even the power of your newsletter. All you need to do is not send the email(s) you want to test to a portion of your list for a period of time and then compare the results — did your triggered emails, new drip sequence, or newsletter actually have an impact on conversion? Or did the control group purchase just as much as the test group?
Obviously, the longer you run your test and the greater percentage of subscribers you have in your control group, the more accurate your results will be.
Now, let’s discuss how you can set up and run your first control group test, some best practices, and a few things you should keep in mind.
Find out what impact your email marketing actually has on customer behavior
The first step to running your control group test is determining what you want to test.
Here are a few ideas:
- Your email marketing efforts as a whole and its impact on revenue.
- The buying-impact of your abandoned cart email.
- The buying-impact of a coupon offer.
- The long-term buying impact of your newsletter.
- The long-term buying impact of a specific type of email (sales, drip sequence, triggered, etc)
What you choose to test will determine how you need to test it — how big your control group should be and how long your test should run.
If you’re going to test the overall impact of your email marketing efforts on revenue, then I recommend setting aside no more than 10% of your email list as the control group and only running your test for a couple of weeks. While this will provide you with slightly less accurate results, it will ensure that you don’t harm your relationship with your subscribers, which is priority #1.
In order to increase the accuracy of your results (without harming your subscriber relationship), you can do this several times throughout the year with randomized control groups and collect the cumulative data at the end.
That way, none of your subscribers are left behind for too long and you do the testing in bite-sized chunks.
Which reminds me, for any control group test you run, the subscribers on your control group should be completely random. Don’t hand-pick subscribers or that will skew your results.
If you’re wanting to test something less significant, like the buying-impact of your abandoned cart email, then you might consider selecting a control group that is up to 50% of your email list and running the test for a month. That is, you’d stop sending your abandoned cart email to a randomized 50% of your list for a month. This will give you great results and it wouldn’t damage your relationship with subscribers since you’re only withholding one triggered email.
Whatever you decide to test, make sure that you run the test in a way that doesn’t harm your long-term relationship with your subscribers.
That should always take precedence.
Conclusion
Now, you know how to run a control group test and determine the buying-impact of your email marketing efforts.
And it’s not nearly as difficult as many marketers think it might be. All you need to do is stop sending emails to a portion of your email list for a period of time and then measure the activity of those subscribers up against those who did receive your emails. What percentage of people purchased?
Finally, you’ll be able to present an accurate picture of how much revenue your email marketing efforts (and even specific email campaigns) are generating for your client’s business. No more guessing or using confusing attribution models, just cold-hard, statistically meaningful data.
