Scaling from a test to an always-on programme
A test that worked establishes that the channel can work in the tested conditions. Scaling changes several of those conditions at once, and a programme that assumes the test result transfers unchanged will be disappointed.
Knowing what changes is what makes the scale-up survive its first review.
What changes about the stores?
The test used a subset, frequently the better ones. Scaling adds stores with lower traffic, weaker category performance and more variable execution, so the average effect falls even if nothing else changes.
That is dilution rather than failure, and expecting it prevents the scale-up being read as a decline.
What changes about execution?
Consistency. A test is watched; a programme is not. Screen uptime, creative refresh and store list maintenance all degrade without somebody owning them.
The operational discipline that made the test work has to be built into the programme rather than assumed.
What changes about creative?
Wear-out becomes the binding constraint. A test running one execution for a few weeks does not exhaust it; an always-on programme running the same execution for a year certainly does.
A rotation calendar with an owner is the requirement, and it is the element most often omitted from a scale-up plan.
What changes about measurement?
Holdouts become harder to justify commercially and more necessary analytically. A programme running everywhere has no comparison, and its effect becomes unmeasurable exactly when the spend is largest.
Retaining a rolling holdout is the standard answer and it needs agreeing before the programme rolls out.
What changes about the store relationship?
At scale, the number of operators grows and the average engagement falls. Screens switched off, moved or obstructed become a material delivery problem rather than an occasional one.
That is a field and operations question with a direct media consequence, and it should be resourced as part of the scale-up.
What should be built before scaling?
A store list maintenance process, a creative rotation calendar, a measurement design with a rolling holdout, and a reporting pack with owners. Four things, and their absence is the usual reason a programme's second year looks worse than its first.
How fast should a scale-up go?
Fast enough to be worth doing and slow enough to keep the cohorts comparable. Rolling out in waves allows each wave to be compared against the last, which is how execution quality is monitored at scale.
Frequently asked questions
Should I expect the test effect to hold?
Expect it to dilute as weaker stores are added, and plan against the diluted figure rather than the test one. A business case built on the test effect applied to the whole estate will miss.
Do I still need a holdout?
Yes, and retaining a small rolling holdout is what keeps the programme measurable. Giving it up at scale is the decision that makes every subsequent review an argument.
How often should creative refresh?
Frequently enough that a regular shopper is not seeing the same execution for months. A calendar agreed in advance is what makes it happen.
What about store churn?
It becomes a maintenance task at scale. A store list that is not maintained drifts away from the distribution list, and spend goes to stores that no longer stock the product.
Who should own the programme?
One named person with the operational elements as explicit responsibilities. Ownership spread across media, trade and field produces a programme nobody maintains.
How should waves be structured?
By cohort, so each wave can be compared against the previous one. That comparison is the early warning on execution quality.
What is the most common scale-up failure?
No creative rotation, followed by an unmaintained store list. Both are avoidable and both are invisible until a review.
Does cost per outcome change at scale?
Usually it worsens somewhat as weaker stores are added, which is a normal and predictable pattern. Building it into the plan prevents it being read as a problem.
What is the single most important thing to retain?
The measurement. A programme that stops measuring when it scales cannot defend itself in the year it most needs to.
### What should the first annual review cover?
Delivery against plan, creative refresh actually executed, store list currency, and effect against the rolling holdout. Four checks, and each corresponds to an operational element that degrades without attention.
Reviewing them together shows whether a decline is an effect problem or an execution problem, which are different and call for different responses.
Most second-year declines in this channel are execution rather than effect, and the review is what distinguishes them.
How large should a rolling holdout be?
Large enough to measure and small enough to be commercially tolerable, rotated so no store is permanently excluded. The size comes from the same power calculation as any test.
Should the programme have a stop criterion?
It should, agreed in advance, in the same way a test does. A programme with no criterion for stopping continues by default, which is a decision made by inertia.
What should be resourced at scale?
Store list maintenance, creative refresh, uptime follow-up and the measurement. Four operational tasks, each with an owner, and their absence is why second years disappoint.
How should the business case be built?
On the diluted effect rather than the test effect, with the operational costs included. A case built on the test figure applied to the whole estate will miss. General planning practice is discussed by bodies such as OAAA.
Considering NRS Digital Media? Start at nrsdigitalmedia.com, read the media deck, or contact the team.