Campaign reporting a client can verify
A post-campaign report should let somebody who was not involved determine what ran and what happened. Most reports in this channel are designed to summarise rather than to be checked, which is a different document with a different purpose.
Verifiability is not an accusation; it is the property that makes a result usable.
What makes a report verifiable?
That every figure can be traced to a source a reader could inspect. Delivered plays trace to playout logs, uptime to monitoring, sales effects to a stated analysis on named store groups.
A figure with no traceable source is a claim, however reasonable it looks.
What should the delivery section contain?
Plays by store or store group, uptime, the plan for comparison, and any make-goods applied. Four elements, and they answer whether the campaign ran as sold.
Store-level detail is what allows the buyer to check rather than accept. A network confident in its delivery has no reason to withhold it.
What should the effect section contain?
The design, the store groups, the measure, the window, the result with an interval, and the compliance data. Six elements, and their absence means the result cannot be judged.
The compliance data is the one most often missing and the one that most changes interpretation of a flat result.
What about the creative section?
Which executions ran, where and when, with delivery by execution. That is what allows any conclusion about creative performance, and it is frequently absent even where multiple executions ran.
How should uncertainty be presented?
As a range, prominently. A single lift figure with a decimal place implies a precision that a store-level test rarely supports, and a range invites the right conversation about risk.
That is uncomfortable in a results meeting and it is what distinguishes a report that survives scrutiny.
What should be said about what went wrong?
Whatever did. Stores that did not run, creative delivered late, a competitor promotion during the window. A report that mentions no problems in a physical campaign across many independent stores is not describing reality.
The easiest person for a results deck to fool is its author, and naming the problems is the discipline that prevents it.
Who should the report be written for?
Somebody sceptical. A report written for an audience assumed to be supportive omits exactly the things a sceptic would ask for, and the sceptic eventually reads it.
Frequently asked questions
What granularity of delivery should be provided?
Store-level, or store-group level at minimum. National summaries cannot be reconciled against anything and cannot support a measurement analysis.
Should raw logs be included?
A summary in the report and the underlying logs available on request is a reasonable arrangement. The availability matters even when nobody asks.
How should a flat result be reported?
With the compliance checks, the detectable effect size and a specific conclusion. A flat result presented as a failure without those is an interpretation rather than a finding.
Who should write the effect analysis?
Whoever has the data and the method, with the design agreed in advance. An analysis designed after the result is known cannot be relied on regardless of who produced it.
What about third-party verification?
For high-value campaigns it is available in the wider out-of-home market and it is worth considering. Standards work is developed by bodies such as the IAB and out-of-home measurement bodies.
How quickly should a report arrive?
Soon enough to inform the next decision, which is usually weeks rather than months. A report arriving after the next buy is committed has no function.
Should the report recommend anything?
A recommendation is useful and it should be separated from the findings so that the findings can be accepted independently.
What is the most common weakness?
Reporting impressions without delivery and effect without compliance. Both make the numbers unfalsifiable.
What single element would improve most reports?
Delivered plays by store next to the plan. It answers the first question anybody should ask.
### Who should the report be reconciled against?
The plan and the invoice, both. Those three documents describing the same campaign should agree, and reconciling them once per flight is the check that almost nobody runs.
Where they disagree, the disagreement is informative rather than accusatory. Most discrepancies have an explanation and the explanation is worth having on record.
Should the network's analysis be accepted as final?
It should be reviewable, with the design agreed beforehand and the data available. Accepting an analysis whose design was chosen after the result is a weaker basis for a decision than the effort of agreeing it earlier.
### What should the reporting cadence be?
Weekly delivery during the flight and a full report afterwards. The weekly view allows a shortfall to be remedied while it still can be; the final report is the record.
Most networks can provide weekly delivery and most buyers do not ask, which is why so many shortfalls are discovered after the campaign has ended.
Should the report include the analysis code or method?
Enough method that the analysis could be repeated. A result described only by its conclusion cannot be checked by anybody.
Who should receive it?
Whoever will make the next buying decision, which is not always the person who commissioned the campaign. A report that stops at the media team has not reached the decision.