The Ultimate Guide To Split Testing

When creating landing pages, it is not uncommon for marketers to decide that they know what it should look like based purely on intuition. Unfortunately however, when marketing decisions are based on intuition and not hard data the results can be disastrous. This is why split testing is necessary.

In order to run effective campaigns, you need to make marketing decisions that are based on conclusive research. I mean, you will need to run conversion rate optimization tests to determine what helps you achieve the results you are looking for.

Conversion optimization testing is also referred to as A/B testing.

Conversion optimization tests are essential as different demographics will have varying response to different marketing campaigns. Therefore, it is always wise to find out which marketing campaign to execute for a particular audience.

One challenge with running conversion optimization tests is that the results can be a bit convoluted making it easy to make mistakes in other aspects of your marketing strategy. You need to understand how to effectively conduct CRO tests in order to avoid this.

Defining Conversion Rate Optimization Tests

There are different types of conversion rate optimization tests, in this article however, I will only focus on the most popular, that is, A/B testing.

 

With split testing, you are testing different types of marketing material with a select group of audience and collecting data on how the audience responds to the marketing. Using this kind of test, you can determine which button color on the call-to-action button has the higher click rate.

A/B testing is time-consuming which discourages many from doing it. However, the data collected in the end can have a make or break impact on your overall marketing campaign so I encourage you to try it out.

Basically, when doing a split test you will need to have two versions of the same piece of content. The versions will be similar in every respect save for a few minor alterations. You will then direct traffic to each of the landing pages and find out which one performs better.

Now that we have that out of the way, it is time we discussed how to efficiently set up, run and measure a split test.

Conducting a CRO Test

Before setting up the split test, you need to determine which variable you will be testing. Even though you might want to test a number of variables, you should only test a single variable at a time, otherwise you will have a hard time determining which of the changes you made led to a spike in sales or lead generation. This single variable you will be testing is known as the independent variable.

Some marketing aspects that benefit greatly from split testing are emails as well as landing pages. Some of the features you might want to split test include:

• Layout
• Wording
• Design
• Subject lines
• Sender names

You need to understand that the changes you make in a bid to get better results don’t have to be extreme. Simple changes such as image modification or the words that feature in your call to action can result in a marked improvement.

If you think your marketing campaign would be better served if you tested more than one aspect, then you should try out multivariate testing which I will write on later.

Identifying Your Goal

For each test you make, you will need to measure a couple of metrics, however, it would be wise to zero in on a particular metric to focus on. You should do this even before setting up the test. This primary metric will be your dependent metric that is, the main reason why you will be setting up your split test is to boost the chosen metric.

Before running the CRO test, have an estimate of how well you expect for the dependent metric to perform then check the CRO test results against your anticipated results.

If you don’t do pick a dependent metric at the start of your CRO tests, chances are you won’t have set up your split tests in the most efficient way.

Create a Challenger and a Control

After identifying your independent and dependent variables as well as achieving your desired outcome, use the collected data to create a version of what you need to test as your control.

A control is version A of whatever you are testing while the challenger is the variation of whatever you will be testing, that is, a landing page or an email.

For example, let us say you are not sure you know whether your landing page would perform better with or without testimonials.

Set up the control page without testimonials and create a challenger page with testimonials.

Divide Your Sample Groups Randomly and Equally

When conducting CRO tests with audiences that you have control over such as people on your email list, you will need to use more than one type of audience in order to have conclusive results.

There are various tools that can help you achieve this such as Hubspot Enterprise.

Sample Size

The type of split testing tool you use will determine the sample size in your CRO tests. If you are A/B testing an email you might want to send to a smaller portion in order to acquire statistically significant results.

On the other hand, if you are testing an aspect that does not have a finite audience, such as a web page, then the sample size will be determined by how long you run the split test for. In this case you should run your test long enough to achieve a significant number of views.

Run one test at a time

By testing more than one thing in an A/B test you will end up with complicated results. For instance, if you test an email campaign that directs traffic to a landing page you are testing at the same time, it is near impossible to know what modifications on what aspect, that is, email or landing page that led to the results you have.

Test Variations at the Same Time

In order to know what variation gives the best results, you need to test both variations simultaneously. If you tested them at different times, it would be difficult to know which one was the best performing as time of day, week or month might be a factor affecting the performance.

Perhaps the only exception here would be when you are actually testing the best time to direct traffic to your offer. You can do this with emails to find out when is the best time to send emails to your list. The best time for sending out emails for the best results is based on other variables as well other than time, that is, subscriber engagement, target market etc.

Let the Split Test Run for a Significant Period

When conducting split testing you need to make sure that the tests run long enough for you to collect significant statistical data showing which of your two variations gives the best results.

If your business does not receive a lot of traffic, it will be a while before you have sufficient data to make a decision. The converse is true.