I’m sure the title of this post is going to make a few folks upset. Don’t worry, the content is going to really make you mad. Not at me, but maybe at yourself, and especially at whoever taught you how to split test.
That’s because, for most Adwords advertisers, the title of this post is very accurate. Most advertisers are doing split testing all wrong.
If you’re doing A/B, you’re robbing yourself of profits at every step.
That’s because, in a typical A/B split test, what you’re doing is keeping your Control (the best performing ad) running, and creating a new test ad to run against it. Doing it that way is perfectly normal, but it’s also completely, utterly, and totally wrong.
Most of the time, these test ads fail to beat the control… and predictably so. Your Control is the Control because it’s doing well. After a few rounds of testing, it becomes less and less likely that your test ad will beat the control.
The “usual method” of ad testing delivers invalid results – here’s why:
- The Control has a strong performance history, but your test ad has no performance history.
- As a result, your test ad may be ranked lower on the page, leading to lower CTR and (quite possibly) conversion.
In other words, your test ad might have been better – but it didn’t get a fair chance! This means that you may have already rejected many great ads, because you didn’t test them correctly.
But that’s not the worst part! “Control vs. test ad” also loses money:
When you run two ads in equal rotation, your Control is only getting half of the available ad impressions.
When a test ad fails fails to deliver good click-through and conversion rates, you’ve just given up as much as 50% of the profits that you would have had, if you had just left your ad group alone.
Wouldn’t it be better if you could construct a valid test that didn’t bleed money? Here’s how it’s done…
Step 1: Set Up A Valid Test
Instead of running your test ad against the Control, you create multiple copies of your Control.
How many copies you make is determined by how many of the ad impressions you want to go to the test ad.
In the example above, we’ve created 3 copies of the Control, left the Control running, and set up one test ad. Therefore, the test ad only gets 20% of the ad impressions.
This method leaves far less of your profit at risk.
It also allows you to run a valid test, by comparing the performance of your test ad against the copies of the Control only.
Since none of the ads in the “testing pool” has performance history, you can have far more confidence in the test results.
This method has reversed the results of many “close misses” I had with the old method of testing, which I was using myself until 2006.
Step Two: End Test (Fail) or Ramp Up To Full Test
In one of my many “hallway conversations” at the last Stompernet Live event, Andy Edmonds (Chief Scientist @ Stompernet and a real expert in testing) pointed out the real advantage of this method… which is that failed tests usually “fail early.”
Scientific testing types have a much more formal language for this, of course, but I got tired head when he explained that part of it.
I’ll just summarize the main points for you:
- You can usually kill your test ad pretty early because most test ads will be obvious failures.
- Running 3 copies of the Control actually accelerates your testing most of the time.
- If the test ad appears to perform well, you can progressively eliminate copies of the Control to ramp up the pace of the test.
- The final test is the validation test, where you run your Test Ad vs. Control – but now, you’re doing it right!
Since I’ve started doing this “Andy’s way,” my ad tests are running even faster than before. Way cool.
But what’s the real goal of all this testing? Higher click-through rates? Lower cost per click? These statistics are merely means to a greater end, my friend, and they can often lead you astray.
Indulge me for a minute, please. I didn’t get to be the “Keyword Guru” for nothing, you know…
There’s one other huge mistake that most folks make when they’re split testing ads - they don’t compare the business performance of their ads, just the click-through rate.
The problem should be obvious – you aren’t trying to buy traffic, you’re trying to buy customers. Actually, that’s only sort of true. What you’re really trying to buy is profit. Specifically, what you really want to achieve is the maximum possible profit per keyword.
At Stompernet, we call this statistic “Profit Per SERP”
Understanding how much profit you make every time one of your targeted keywords gets searched is a powerful idea. One of the most important metrics that any enlightened website owner will learn to measure and improve. It’s the KEY to search marketing success.
You have a lot of control over every variable in the search marketing equation, except for one thing…
You can NOT change the number of times people carry out a given search.
During the month of May, some number of people are going to search for “mortgage rates,” or “baby gifts,” or “free seo book.” We are powerless to change this number.
The number that we’re trying to change is our average profit for each of those searches.
By doing Adwords right, you can change that number, and in doing so, change the rules of the game.
We’ll talk more later.
For now, I’d like to invite you to watch the 50 minute instructional video that I prepared for you, with the help of master video editor Andy Jenkins. Watch online or download the Quicktime – but do watch… you don’t want to miss this.
