Measuring media effectiveness is essential for marketers. However, as people adjust to the realities of our new normal, measurement has become increasingly difficult. Campaigns that deliver a healthy return on investment today may not work as well tomorrow.
In a constantly shifting and uncertain environment, businesses face an even greater need to make careful, impactful business decisions. Yet marketers must also understand how consumers react on a nearly real-time basis. As the path to purchase changes, marketers must be agile enough to adjust and optimise campaigns to capture customers in the right moment.
One of the best ways to know if your marketing is working is by running experiments. Marketing experiments can be powerful tools to determine how many conversions each media type drove as a result of a marketing message — typically called incrementality, or lift — and how many would have occurred anyway, which in turn can be used to determine channel-level budgets. These experiments offer marketers a clearer understanding of what’s performing well and what can be improved.
Marketers must understand how consumers react on a nearly real-time basis to adjust and optimise campaigns.
Whether incrementality experiments are the right tool for you, and how deeply you should invest in them, depends on the realities of your business situation. To understand which activities are most worthwhile, start by evaluating your stage of business recovery, and consider the costs and benefits of different testing strategies.
3 modes of business recovery and measurement
There’s no one-size-fits-all measurement approach for these fast-changing times. The right strategy depends on your goals, your customers, and the marketing conditions in your industry right now.
To help you assess your best measurement options, and the pace at which to implement marketing experiments, we’ve divided businesses into three broad modes of business recovery.
Crawling to stability
Perhaps your industry has been negatively impacted by the pandemic and is recovering from a loss of revenue. If consumer behaviour is still evolving rapidly — like that around air travel — it may not be a good time to run a full-fledged incrementality study. That’s because any insights you gather today may lose value as time goes on.
Instead, optimise campaigns for the short term, maximise immediate cash flow, and look to boost conversions quickly. Quick learning can be impactful for revamping your marketing strategy or developing new creative approaches. Businesses that have sufficient volumes of data can use data-driven attribution to learn which channels, formats, and campaigns play the biggest role in helping achieve marketing goals. By connecting data-driven attribution to automated bidding, you can measure changing consumer behaviour and meet customer demand in real time.
Walking on the mild side
Industries experiencing volatility should start preparing for recovery, while proceeding with caution. Consumer priorities that have shifted as a result of the pandemic, such as ordering takeout instead of dining in, could one day shift back. Short-term KPIs, and anything you learn from them, could be less relevant in the future. However, building organisational agility now can help you prepare for what’s next and drive growth in the long run.
Focus on improving existing campaigns and increasing volume from your current customer base through quicker optimisation-focused tests and experiments. Set up a principled approach for testing and analysing results, for example, as a weekly pulse check on sales stability. Consider using tools that help you run A/B tests on a portion of your ads to see how small changes to your campaign, such as ad copy, keywords or a landing page, impact your results. As your marketing experiment runs, you can monitor and compare its performance against your original campaign and roll-out changes that are performing well.
Consumer priorities that have shifted as a result of the pandemic could one day shift back.
Running at full speed
Some industries have experienced a surge of revenue due to stay-at-home orders. If your business is dealing with increased demand, running incrementality experiments can help you understand the true value of your optimised marketing strategy, inform channel-level budgeting, and gain causal insights.
Since you’re likely appreciating stable sales growth, you can run multiple incrementality tests, including user-level and geo-based studies. For example, to evaluate a new YouTube campaign strategy, you can run Brand Lift and Conversion Lift tests to measure the full-funnel impact, from ad recall through to conversions. You can also use these insights to find out which creative and audience strategies are most effective in driving lift.
Geo experiments are a useful alternative to user-based studies if you want to evaluate the impact of your ads on in-store sales, or if you’re looking for a fully transparent and replicable methodology. These tests divide the country into geographical regions, and then randomly assign each region to either a test or a control group. By showing ads in one region and not the other, you can measure whether your marketing is having an impact. For the validity of a geo experiment, it’s important to avoid any factors that could unilaterally influence the test or control group, for example, store closures in a test region only.
Keep in mind, because different geographies have adopted different responses to COVID-19, brands should have at least one month’s worth of stable data for both regions before running any tests. Given uncertainty regarding baseline conversion rates and effect sizes, be conservative when estimating the power of these studies.
No matter which stage of recovery you’re in, businesses navigating new media plans and measurement tools should start simple.
When running any tests, be mindful that performance may be unusual, and consider how likely test results can be generalised to business-as-usual contexts. If you’re investing more than usual during this period to drive growth, remember that the extra marketing investment may result in lower incremental returns on investment. Test again once your marketing budgets are back to their usual levels.
It’s not business as usual
In a changing environment, marketers need to listen more closely to customers than ever to make each dollar count. No matter which stage of recovery you’re in, businesses navigating new media plans and measurement tools should start simple.
Continue to use your organisation’s sources-of-truth measurement strategies. Once you’ve optimised your campaigns within the new normal and are experiencing performance stability, marketing experimentation is a great way to validate and increase the efforts that deliver growth for your business.
