A Complete Guide to A/B Testing and How It Works

What is A/B testing?

Netflix… Booking.com… Facebook… Do they look the same as they did last year? Two years ago? 

No, of course they don’t. 

These companies have used A/B testing since they first began to create a better experience for the user. 

By listening to what the user wants and trialling and testing each change as they go.

These companies test everything from the colour of their CTA (call to action) buttons to the placement of titles, font, and layout. 

Chances are you have been part of their A/B tests without even realising it. 

If your business isn’t doing A/B testing with your website. Or if it’s not part of your marketing plan, then you could be losing out on traffic, followers, and leads. 

A/B testing is beneficial for every company no matter what industry you are in. 

Each time you create a different piece of content, it likely follows the same guideline. For instance, you may have a particular style or use colours, layouts and themes unique to your brand. But how do you know if these colours or themes are actually converting customers?  

For example, you can have two website homepages with different heading images, or different titles, etc. But how do you don’t know which one would appeal more to your viewers? 

You test them using A/B testing to see which gets the most interaction. 

This way you can create the ultimate user experience and design which will gain you the most traffic or leads. 

Let’s imagine you have a landing page and you can’t decide which image is best to use. 

So you create two versions: one with one image and one with the other. You send one version (control) to 1000 people. And the other version (challenger) to another 1000 people. 

Similar to a scientific experiment, you should try to test with as many people as possible. That way, your results will be more accurate and measurable.

After conducting your experiment, what’s the result? 

One of the landing pages has no interactions at all. Whereas, the other has click-throughs, shares, traffic – you get the idea. 

That means from now on you can pick similar images and themes knowing your audience will like them. 

With A/B tests you can explore different areas of your marketing plan. Not only can you find out which actions align with your viewers. But you can see what works and what doesn’t. And this creates invaluable insight about your audience. So you can make smarter, more profitable decisions for your business. 

Of course, it would be nice to test your whole webpage or marketing plan with your audience. However, this would take up too much of your valuable time and resources. As such, it’s best to stick with testing small elements, one at a time. 

You might be thinking: well that seems pointless. Trust me, it isn’t. Changing just one element such as the colour of a CTA button can have a big impact on your conversion. And over time, all of these smaller changes compound. So that eventually, your pages will perform significantly better.

Bear in mind, though, that some elements such as a single word or sentence don’t make much of a difference. So changing these would probably be a waste of your time and you should prioritise according to impact.

As such, it’s best to stick with the bigger elements. 

Some of the best elements to A/B test are:

•  Headlines and copywriting
•  CTAs
•  Images or videos
•  Subject lines
•  Content length
•  Product descriptions
•  Email marketing
•  Landing pages
•  Website forms and pop-ups
•  Social proof

At its core, A/B testing helps you to see what you are doing well and what can be improved. This can be with words, images, fonts, content length, CTAs and everything else you use on your website. 

In fact, the simplest of changes can cause a huge increase or decrease in conversion rates. For instance, changing a CTA button from blue to red. Although this may seem like a small change, it really can make a big difference. 

Once you have identified what factors your audience responds to best, you can build on them. But if you never try, you will never know! 

A benefit of A/B testing is that it can identify any holes in your content. And it can highlight previously unseen factors that may be hurting your conversion. 

As such, testing in this way can fix your conversion rate, bounce rate and also minimises guesswork. Making your website and its content more efficient. 

In addition, A/B testing provides you with facts to back up why you are doing certain things. Particularly useful, if you need to persuade decision makers in your business.

Ultimately, you are using A/B testing to create the best user-friendly experience you possibly can. 

Maybe you want to see if your new CTA button is getting more traffic when placed differently. For instance, at the top of your page rather than in the middle. 

A/B testing provides real audience insights. As well as accurate data about which placement is better, by showing how many click-throughs each version. From there, you can adjust your page depending on the results. 

You can conduct A/B testing in many ways depending on what you are trying to test. For example, if you want to see what subject line attracts more people to open an email. You want to test a question versus a statement. So you send an email with a question subject line to 50% of your mailing list. And another with a statement subject line to the other 50%. 

After a period of let’s say two weeks, you can compare the opening rates of each version. Which means in future you will know which type of subject line to use for your emails. 

If you want to start A/B testing, follow our 7 steps:

First and foremost, before you do anything else, you need to do your research. So it’s time to unlock your inner Sherlock Holmes and think like a detective. 

To begin, find and investigate everything that is currently happening with your website. This includes: 

•  The amount of traffic
•  Backlinks
•  Bounce rates 
•  Click-through rates
•  Identifying key landing pages 

Everything you could possibly find out about what is currently happening. 

Without knowing the current state of your site, you won’t be able to measure your results. You’ll be unable to tell if your testing is improving or making a difference to your site. 

You can conduct lots of tests and have the best A/B testing going. But if you don’t have anything to compare it to, all your efforts will be wasted.

Next, you might find that you have a whole bunch of variables or elements you want to test. However, it’s best to go through them one at a time. 

Identify which is the most important and start there. 

By just choosing one variable you will easily be able to see if it’s made a difference. And you can evaluate the results before moving onto the next variable. 

For example, say you have a landing page which is an article. 

It has a header image, a CTA at the bottom, and is 3500 words long. 

You want to test whether the length of the article is too long. And whether you should move the call to action up to the top. As well as whether you should use an image with graphics or without. 

Identify which one you would like to test first before testing the others. 

You could start by testing the CTA. Once you have finished testing this, you can use the results to make the necessary changes. Then, move onto the images, then content length etc. 

As you may have gathered, A/B testing is all about making decisions! 

But perhaps the most important is about what you want to achieve or improve. For example, let’s take the previous example. We are going to move the call to action to the top of the page. And we want to A/B test whether this is a better location than at the bottom of the page. 

With this testing, you would expect higher click-through and conversion rates from the CTA when it’s placed near the top. This is what you will be looking out for in your testing results. Which location has the higher conversion rate? 

How does this relate back or align to your business or marketing goals? Ask yourself these questions and write down the results.

Testing without an aim or objective in mind is pointless.

Next, you need to create your new variable or element. Basically, whatever you are deciding to test. 

Again, let’s go back to our earlier example of moving the call to action. My new element would be a copy of the same webpage, only with the call to action moved to the top of the page. This is now my “challenger” and the existing webpage is my “control”. 

So, you’ve created your new element…what happens next?

The majority of A/B testing requires the use of specific A/B testing tools. Be warned there are lots of different options to choose from. And some are more budget-friendly than others. 

Here are some good options to get you started:

•  Optimizely
•  Google Analytics & Google Optimize
•  VWO
•  Zoho PageSense
•  HubSpot
•  Leadformly
•  Unbounce
•  Convertize
•  Convert
•  Omniconvert
•  Crazy Egg
•  AB Tasty

The testing tool which is best for you and your website will depend on your business. Consider your needs, budget, existing resources and what you are looking to test. 

Next, it’s time to run your test!

So you’re happy with what you want to test, you’ve set your goals, and chosen your testing tool. Now, it’s time to start your testing. 

Make sure that you run your test for enough time. Not only to get a real feel for the results but to get a good enough sample size. 

Without a large enough sample size, you won’t be able to see a clear difference in the choices you made. 

How long you run your test for depends on the amount of traffic you usually receive. 

If your website gets a large amount of traffic, then you should quickly be able to see a difference. 

On the other hand, if your business doesn’t get a lot of traffic, it could take a little longer.

After your test has finished it’s time to ask yourself a few questions:

1.  Refer back to your goal, are the results what you expected?
2.  Are your results enough to justify the change? 

For the second question, you can use an A/B testing calculator.

If you have a clear winner then it’s time to make that change. 

Finally, it’s time to start planning your next A/B test!

A/B testing can be hugely beneficial for your website and business. It can  create a more user-friendly website. As a result, this will gain you more leads and keep viewer retention. 

But A/B testing can be challenging and time-consuming to conduct. This is due to all the research, and constant evaluation it requires. However, it should still form part of your overall marketing strategy.

This is because the changes you implement are guided by your audience. As such, your website will provide them with a positive experience. Thus, keeping your site users loyal to your brand and encouraging them to return. 

A meaningful user experience helps you to define your customer journey. Understanding your audience is the best way to keep your website optimised for success. And A/B tests can help you unlock this valuable knowledge and lead you to more website growth, leads and sales.

Many companies lack the time or resources to conduct detailed A/B testing. That’s why often, they’ll turn to a marketing consultant, to take over this process, such as myself. 

My team and I can help you conduct A/B testing and enhance your website, to appeal more to your customers. And we can help you devise a robust marketing strategy that aligns with your business goals. 

Why not get in touch with us for an obligation-free chat. We can help take your business to the next level and get you the growth you deserve.