Creative testing on in-store screens

Which of these two ads would you run? Most teams answer with opinion, and opinions differ. Creative testing replaces the debate with evidence: run two versions in comparable groups of stores, measure the same outcome in both, and see which one did better. On in-store screens, where the outcome is often store-level sales, that evidence can be clear.

It can also be misread. A good test is as much about reading results honestly as about running them.

What should you test?

One thing at a time. A headline, an image, static versus motion, offer-led versus brand-led. If two versions differ in several ways, you won't know which difference mattered.

Things that would change your decisions. Test elements where the answer affects future creative. If the team will always use the same pack shot, there's no point testing it. If there's genuine uncertainty about whether to lead with price, test that.

Things with a plausible effect. A tiny change in background shade is unlikely to move sales measurably. A change in the main message might.

How should stores be split?

Into matched groups. The logic of A/B testing applies: each version runs in a group of stores that's similar to the other on size, location type and recent sales of the product. The closer the match, the more confidently you can credit differences to the creative.

Check the match before the test. If both groups sold similar amounts of the product for several weeks before, the comparison is fair. If one group was already selling more, adjust the groups.

Keep everything else equal. Same dates, same product, same distribution, same offers in store. Any other difference weakens the test.

Ask the NRS Digital Media team how creative versions can be assigned to store groups in your campaign.

How do you read the results honestly?

Look at the size of the difference, not just its direction. If version A outsold version B by a small amount, that may be noise. Store sales vary week to week, and a small gap can appear by chance.

Look at the range. A result with a wide range of possible effects is less certain than one with a narrow range. If the range includes no difference at all, the test hasn't picked a winner.

Look over enough time. A test that runs for one week can be dominated by a single unusual day. Several weeks give a steadier read.

Store-level scan data, such as the data analyzed by NRS Insights, is the kind of source that supports this sort of store-group comparison.

What are the common mistakes?

Declaring a winner too early. Stopping a test the moment one version pulls ahead invites false conclusions. Decide the test length in advance.

Testing too many things. Running five versions at once splits stores into small groups, which makes each result noisier.

Ignoring null results. A test that shows no difference is useful. It means the choice between versions probably doesn't matter much for that outcome, and the team can decide on other grounds.

And generalizing too far. A result for one product in one season in one set of stores is evidence for similar situations. It isn't a universal rule. Keep a log of tests and results, and check the creative specs page before building test versions.