Split testing is an essential skill for an AdWords Expert professionally managing an account in today’s world. In this article we are going to explore some of the details that need to be considered to run a professional split test. We could write an entire book on this topic and still not cover it all, so for this article we are going to limit our scope to split tests conducted in the search network of Google AdWords using text ads.
First thing to consider is the goal of the split test; they can be designed to:
- Learn something new
- Experiment with a break through idea
- Test the Margin of Error
Why do we bother with split tests?
Splits help us test what we think is true against what in reality is how our specific market is reacting. With over 20 years in this field I cannot tell you the number of times that market response has surprised a wide range of experts. One thing that we know for sure is that split test results are clues, not facts and it takes lots of tests to increase the probability that the item being tested and the results being seen are the same thing.
A/B Tests – Learning Something
Technically, all split tests that run 2 ads are an A/B test. For the purpose of this article, an A/B test is one that is testing a single variable. This results in ads that evolve in small steps because you can only change one thing at a time. To reiterate, CHANGE ONLY ONE THING per test. Using an A/B strategy requires a long term view and the will to stay the course. If you are part of the instant gratification society it is unlikely you will be successful with a true A/B approach.
Multi-variant – swinging for the fences!
When most people talk about split tests this is the type of test they mean. They write completely new ad copy and put it in competition with the current winner. The challenge is that when the new ad wins there are so many differences that you have to guess at what caused the result. This is not to mean that this does not have value because it does. If what you need is a breakthrough ad then this is the way to run that test, but you have to accept the fact that this the equivalent of swinging for the fences. If it works, it is a home run but it is also risky and likely to result in a strikeout. When you are trying to find the message that resonates with your audience, this is a valid strategy. As the account matures it might make less sense.
Cross Traffic Tests
Sometimes an account simply lacks the volume to do a test in a reasonable period of time but has enough traffic at an account level. In these cases you can sometimes engineer tests that tie all or most of the traffic together. The key to this, is creating ad copy that is generic enough to run the same headline or body in all the groups. The compromise here is that the more generic the copy the weaker the ad tends to be, so getting the test at all costs can sometimes be too much. While the headline is giving up too much the body text can be tested.
Margin of Error
A margin of error test is exactly what it sounds like. Everything in split testing is a clue not a fact and you have to understand the range of accuracy you are dealing with. In a margin of error test, you run two identical ads and run them against each other. This shows you how much the data might be off, based on no variables that are under your control. When this is done several times, a range of error will be visible that you can use to decide how probable the result will repeat. Over the years we have noticed that in most accounts this is about plus or minus 10% of the rate being evaluated. So an account with a normal 2% CTR could come in from 1.8% to 2.2% simply from the margin of error. If the result of a test is within this range then it is too close to call.
Common Mistake 1 – Confusing clues with facts
Clues are data that indicate support for a theory but are not proof. A fact proves that a theory is universally true. In marketing clues are common and facts are exceptionally rare. How rare you ask? Good question: in the last two decades in digital marketing, I have yet to find a fact. As you build the case for your decision the more clues, the more likely what you believe is true.
Common Mistake 2 – Not considering outside events
AdWords does not operate in a vacuum. The best example I can give to you is a snow removal business that wanted more sales leads. No matter how much they advertised and promoted the business, they just could not move the needle. Then a huge snow storm came and the phones lit up like a Christmas tree. The truth is that search marketing serves the demand but almost never creates demand. This is simple to understand when you stop and think about the first step in search, which starts with a need, then a search query, and finally the search engine results.
Common Mistake 3 – Thinking that the body text counts
In most cases the body text (two lines under the headline) simply do not change the results. We think the reason is that people do a search, see the headline they need, stop reading, and click on the ad. We have tested this many times and changes in the headline create meaningful change in the traffic but the body text does not. We often see people struggling for hours to create the best copy possible often for little or no return for their efforts. Invest your time and testing on the headline and write good informational copy for the percentage of the audience that do read those lines.