Most brands test like this.
Someone in a meeting asks what we should test next. The team picks something. A new headline. A different image. A layout change.
They run it. One wins. They roll it out.
Next quarter it loses. Was it the creative? The time of year? The channel? Nobody knows.
Same meeting. Same question. Start again.
That is not testing. That is a lucky dip.
One question changes it
What are we trying to learn?
When each test is designed to teach you something specific about the people you are trying to reach, the learnings compound. You stop guessing what works and start building a picture of what makes people respond.
The difference is between testing different executions and testing different arguments. Most teams do the first and think they have done the second.
Different headlines saying the same thing is not a test of anything. Different reasons someone should care is.
An example that caught me out
I ran a test for a national retail brand last year. Loss framing against gain framing.
I expected loss to win comfortably. It is one of the most reliable findings in behavioural science. People feel the pain of losing something roughly twice as much as the pleasure of gaining it.
Gain outperformed loss by over 160 per cent.
Did not see that coming, and that is the whole point.
If we had not tested it properly, we might have run loss framing across future campaigns and never understood why the results were not landing the way we expected.
A good test does not just give you a winner. It gives you a reason. And that reason becomes the next brief.
The moment matters as much as the message
There is a second variable most testing ignores completely.
The same person can care about your cause on Monday and ignore your appeal on Tuesday. Not because they have stopped caring. Because the moment is wrong.
Three days before payday they scroll past. Three days after payday they act. Nothing about their values has changed. Their headspace has.
Ben Briggs and the team at Join the Dots have built an activity calendar for charities around exactly this, looking past the handful of annual fundraising moments the sector keeps returning to. Payday. Moving house. A news story that lands. A change in someone’s circumstances.
It is a charity example, but the principle is not sector-specific. I have been banging on about it with retail clients for years, and testing it regularly has produced year on year uplifts.
The message has to be right for the person and right for the moment they see it.
A control is not a family heirloom
The last blocker is emotional.
Teams get attached to controls, especially ones that have worked before. Which is understandable, and also the reason a lot of testing programmes stall.
A control is useful because it gives you something to beat. That is all it is.
If the challenger wins, you scale the learning. If it loses, you know what not to chase. Either way you have stopped guessing.
The good news is that most brands will not bother doing this properly. Which is exactly why the ones that do win more often.
Book a call