With new technology and ever-shifting consumer behaviours, marketers need to be agile and prepared to constantly experiment.
One way of doing this is to propose and test a hypothesis, then use its results to refine your approach — from creative execution to the tools used. This is the basis of a ‘test and learn’ culture, a critical part of any successful digital marketing transformation.
The foundations of a test and learn culture
Companies with a developed testing culture have an “always on” approach. They carry out research constantly, not just when a problem arises. This helps ensure a continuous cycle of development, where marketing and product strategies are refined based on data.
Here are key ways in which you can start implementing such a culture within your own business:
Make experiments accessible
The best digital leaders minimise bureaucracy and create a culture where any employee with a clearly constructed hypothesis can conduct an experiment.
Adopting a company-wide experiment template will help simplify this process. Employees can fill it out before starting an experiment, outlining the reasoning behind the test and the actions that will be taken for each possible result.
Encourage employee curiosity
The more employees who experiment, the better it is for your business.
A great example of an organisation empowering their staff to be more curious is British Vogue. The team is encouraged to explore performance data and analytics, experimenting with any tools that can’t be broken in order to shift focus from how well something did to how it can be done better.
Make decisions based on data
In an effective test and learn culture, data comes first — often even before the opinions of senior leaders.
In pursuit of scale, fitness brand Gymshark pivoted from relying on the founder’s gut instinct to data-based decision-making. With an overall strategy that builds on the results of experiments, the company is enjoying rapid growth both in the U.K. and globally.
Allocate sufficient budget
Understanding new challenges and opportunities is business critical, especially in unpredictable times. This makes testing an ongoing, necessary process that requires sufficient budget.
A general approach is to allocate resources according to the 70/30 rule:
How to conduct tests effectively
The following four guidelines will help ensure your tests are effective and actionable.
Formulate a clear hypothesis
To ensure a productive experiment, your hypothesis must be clear, testable, logical, and based on data. This usually involves carrying out a situation assessment, brainstorming, or analysing the data from previous experiments to understand more about a particular topic.
When the experiment is complete, it should be absolutely clear whether the original hypothesis has been proved or disproved.
Test one metric, observe them all
Ensure you concentrate on one particular metric, such as revenue, during testing. The way it’s assessed should clearly answer the question posed in your hypothesis.
Alongside this main focus, you can also observe other relevant metrics. These will contribute to a broader picture, enabling you to gain additional insights such as whether optimising a single KPI causes others to deteriorate.
Ensure your team is diverse
The composition of your team plays an important role in the testing process. Keep team numbers small but invite employees with different expertise to increase diversity of opinion. A recommended effective combination would be a product manager, an analyst, and a marketer.
Properly implement your results
Once you’ve analysed an experiment’s results, the next step is to put them into practice. This could be by using your findings to make adjustments to an existing campaign strategy.
Aside from the intended use, all experimental results should form the basis for ongoing research. This will help to improve future hypotheses and allow results to be compared over time.
One example of a particularly effective solution would be to create a searchable database of experiments, so employees know more about any changes that have been implemented — allowing them to formulate their own hypotheses.
Test and learn framework
There are numerous frameworks for an effective test and learn process, and you will naturally refine yours over time. Here’s one example used frequently at Google
Tools that help automate and simplify your testing
- Brand Lift helps you understand how video ads on YouTube impact consumer perception of a brand in real time.
- GeoX measures the incremental value of ads by comparing behaviour in different regions. Campaigns are implemented or intensified in test areas, then the test and control groups are compared in terms of cost per action (CPA) or return on ad spend (ROAS).
- Firebase Engage offers a wealth of tools and features for testing user experience and messaging within apps. A few use cases are outlined here.
- Campaign drafts and experiments allow you to test campaign settings without compromising ad performance. Create a project with the same settings as your campaign, change them, and see how it affects results.
