Marketing is in a state of constant transition. Many of the tactics that worked five years ago—or even last year—no longer produce results. B2B buying cycles are changing. We’ve reached the point of content saturation. Customer expectations are growing and consumers have more power in the vendor-consumer relationship than ever before. Success now requires a keen focus on the customer experience and customer success.
As marketers, we’re charting new territory and are forced to constantly experiment with new, innovative tactics to remain competitive.
Luckily for us, the marketing automation (MA) systems we use provide tools that facilitate this experimentation. An effective way to evaluate new marketing tactics is through A/B testing.
A/B testing lets us deploy two variations of the same marketing tactic, side-by-side, and compare results. In this way, we discover which of the two is more effective. This removes part of the guessing element from our disruptive marketing experimentation and allows us to determine which new tactics to focus on based on data rather than intuition.
Below we dig into A/B testing, discuss when you should use it, and delve into a few best practices for mastering A/B testing in your marketing organization.
What exactly is A/B testing?
With A/B testing, we leverage marketing automation to execute two approaches to the same marketing tactic simultaneously. The best marketing automation solutions let you get pretty granular with A/B testing.
How does it work from a technical perspective?
We can use email marketing as an example to explain the process. When testing two versions of the same email, your MA system will send a sample of each version to two subsets of your overall targeted audience.
Your system will then wait a specified amount of time to measure how each of the two performed. Which had the highest open rate? Which saw the most click-throughs? Which resulted in the most unsubscribes?
Once it has enough data to determine which version is more effective, your system will push that version out to the rest of your target audience.
Why should you use A/B testing?
It’s important to know which marketing tactics best engage your audience, attract new leads, and drive the most lead conversions. If you’re testing an email campaign, A/B testing will tell you which email versions generate the highest open rates, click-through rates, and which generate the most marketing qualified leads.
Experimenting with an email campaign
When applying A/B testing to an email campaign, you can experiment with the subject lines of your emails, the copy of the emails, or the images you use. You can experiment at a more granular level by testing two different font types, font colors, email template designs, headers, sub-headers, names in the email “from” line, and so on.
Testing elements of a marketing campaign
You can also use A/B testing in various parts of a digital marketing campaign. Compare the results of two different landing pages, lead generation forms, or calls-to-action. Moreover, you can test two different marketing campaign sequences to determine the optimal cadence for campaign touchpoints.
Don’t forget statistical significance
When you’re A/B testing new tactics, be sure to apply your test to sample sizes large enough to produce statistically significant results.
If your target audience is 3,000 leads and you only send your initial test versions to subsets of 10 people, your results won’t be reliable enough to represent your entire audience. Sample size (n) is key to effective A/B testing.
Need a quick refresher on statistical significance? Brush up on the subject.
How do you plan & execute an A/B test?
The point of A/B testing is to generate data that leads to actionable insights and empowers you to confidently apply the tactics that are most effective with your target audience. What works for one industry may not work for another.
When planning an A/B test, it’s helpful to follow a set process and stick to it. This produces consistency in your results and strategies. Here is an example of an effective step-by-step process to follow:
1. Define your hypothesis
Determine the question you’re trying to answer. For example, should I send this marketing email from “The [Company Name] Team?” Or, does it make more sense to send it from individual sales executives? You’ll have an assumption of which will be more effective, but that’s just a hunch. Your A/B test will (or won’t) validate your assumption.
2. Determine which & how many tactics to test
Are you going to keep it simple and test two email subject lines? Or are you going to also test send dates to see which day of the week generates the most email opens? If you’re new to A/B testing, we recommend starting by testing one variable, such as an email subject line. It’s best to ease your way into the process and learn as you go.
3. Calculate a statistically significant sample size
Do the math and determine the appropriate sample size for each subset of your test so your results can reliably tell you which tactic to deploy. If you don’t, you’ll be wasting your time because your results won’t accurately predict the results you can expect when you push your tactic out to your entire target audience.
4. Test your test
Quality assurance (QA) is vital to effective A/B testing. Run a test drive of your experiment with some test leads in your CRM database. Be sure you are in that group of test leads so you can walk through the process yourself and ensure everything is set up correctly.
Click every link, complete every form, open every email, and so on. Then check the results to verify that the actions you took are properly represented. If there’s a broken piece of the process, you want to identify it before you execute your test on actual leads or customers.
5. Set your timeframe
How long will you wait, while the test group data is being compiled, before you determine the effective tactic and push it out to your entire audience? The answer is that there is no definitive answer.
The amount of time you should wait depends on how long it will take to accumulate enough data for your results to be statistically significant. That depends on your audience size and how quick they are to act. It’s important not to push out either tactic to the entire group prematurely.
6. Deploy, measure, & analyze
Once you push out the winning tactic, wait an appropriate amount of time, then measure the results. You may find that although Tactic 1 was more effective during your trial test, the results it generated when deployed to the entire audience varied significantly.
If that happens, you might want to run another test, comparing that same tactic with another one, to confirm that it is an effective approach to engaging your audience. There’s no harm in re-testing a tactic because you must understand why a particular tactic was successful.
When should you use A/B testing?
Don’t A/B test any random tactic out of curiosity. You need to set a goal when A/B testing because it is most helpful when you’re trying to solve a problem or improve upon something that’s not working as you need it to.
For example, if conversion rates have been dropping, it’s time to start A/B testing new tactics. If customer retention rates start to fall, pull out your A/B testing playbook. If you simply can’t generate new leads, it’s probably time to experiment with new tactics.
What do you need to conduct A/B tests?
First of all, you need the ability to measure specific metrics—the majority of which can’t be measured without technology. You can’t measure email campaign click-through or open rates without software that automates those processes.
In short, you need a CRM that stores customer and lead data as well as a marketing automation solution with the ability to conduct A/B testing. Some CRMs, like Insightly, include built-in MA capabilities to form a unified CRM system. These are the best kind of solutions to conduct effective A/B testing.
Final thoughts on A/B testing
Now that you understand the basics of A/B testing, as well as why, when, and how to conduct A/B tests, it’s time to get to work. Start thinking about when you might want to run your maiden A/B testing voyage.
If you don’t have the right technology in place to run A/B tests, now’s the time to start thinking about implementing new software—such as a unified CRM—in your organization. Such software does a lot more than allow you to test new tactics. It automates loads of manual processes, ensures data integrity, and allows you to deliver a better customer experience, along with many additional benefits.
If you need to learn more about CRM and MA software, feel free to schedule a free demo with Insightly. We’ll walk you through the benefits you receive from using a unified CRM with built-in marketing automation.
