Measuring Brand Ads Effectiveness in Japan: Striving For Accuracy (Part 1)

There are plenty of reasons why marketers should verify the effectiveness of their advertising and other marketing initiatives, but the main reason boils down to one thing: accountability. After all, marketing managers wielding large budgets need to be held accountable for their advertising and marketing plans. Unfortunately, the way advertising effectiveness is currently measured is not as precise as it could be.

This advertising effectiveness discussion will be divided into two parts: Part one will focus on the underlying accuracy of effectiveness verification, while part two will discuss the agility of effectiveness verification for complex marketing actions.

The surprising bias of online advertising

When measuring the effectiveness of online advertising, biases can easily skew results (such as purchasing intent or brand favorability). This can lead to unintentionally higher or lower measurements. Since it is difficult to create scenarios completely devoid of bias, the real problem occurs when marketers use bias to improve the outcome of advertising effectiveness.

The following are examples of typical biases when measuring advertising effectiveness.

Memory bias: People who favor the brand remember the ad

When the brand preferences of those who remember an ad are compared to the brand preferences of those who don't, results will show that people who remember the ad also have more preference for the brand. Why? People simply tend to remember advertisements for brands they already like. In this case, rather than attributing an increase in consumer favorability to ad exposure, we need to consider that ads are easier to remember when we already have a preference for the brand.

Seasonal bias: Advertising during the right season

Think about it: If an advertising campaign for sunblock is released during the height of summer and purchasing intent is compared in surveys before and after the campaign, the post-campaign period will show positive consumer responses. It’s natural that the demand for certain products such as sunblock increases during particular periods such as summer. This is an example of seasonal bias, in which there is a high possibility that the increase in purchasing intent is actually the result of the season rather than an advertising push.

Media bias: Advertising to users with high information sensitivity

If an advertising campaign specifically targets individuals with high information sensitivity, any increase in purchasing intent or brand awareness is likely to be the result of media bias. Individuals with high information sensitivity are likely to already possess high brand awareness; thus, any positive results from an ad campaign should be disregarded.

You might think that the accuracy of effectiveness verification is greater for online advertising compared to offline advertising, but in reality it’s easier for biases to occur online. For example, in the case of Chromecast—a site which allows people to enjoy online video on their television—ads are targeted at people who are interested in entertainment portals like YouTube. Subsequently, when people exposed to the advertisement are compared to those who weren’t, the influence of media bias is strongly felt. Another example of online bias can be illustrated with Nexus search. When new smartphone models are announced, there is naturally an increase in related searches. Consequently, a Nexus search would integrate advertisements related to the new smartphone to be delivered during this period. Any increase in consumer searches or purchasing intent for the new smartphone should be interpreted as a result of—you guessed it—“seasonal bias” because it reflects an increase in interest on an individual level rather than as a result of increased advertising.

Since the ability to conduct such accurate targeting isn’t possible with television or offline advertisements, biases are limited in these platforms. It’s ironic that the improvement of online targeting accuracy has made it easier for biases to occur.

Removing bias by the targeting ability of digital technology

Online advertising has many advantages.The first is the ability to accurately measure exposure to an ad by recording who was exposed, how many times, and to which ad by using actual records (log data) rather than users' recollections. The second is the ability to accurately target the audience and control which ad is delivered to whom. Although the two advantages of measurement and targeting appear to be different, they’re actually based on the same technology platform. These platforms consist of digital technologies referred to as a Cookie or an AdID, which identify individual users while maintaining anonymity.

When a Cookie or an ADID is examined, it’s not possible to identify the actual individual (such a person named Mr. A). It is, however, possible to measure that individual’s past behavior to determine if he was previously exposed to the brand’s advertisement or if he had ever visited the brand’s website. At the same time, targeting (stopping delivery of an advertisement to a user who has been exposed to the ad five or more times or delivering an advertisement only to a user who visited the brand’s website) is also possible. By combining the ability to measure what was delivered as well as how many times it was delivered, it’s possible to remove the series of biases which exaggerate the impact of advertisements.

First, the conditions in which users were exposed to the advertisement can be accurately assessed based not on actual data rather than their recollections. If people exposed to the advertisement are surveyed after the completion of the ad campaign, their true reactions will be unaffected by memory bias.

Next, to verify effectiveness, it needs to be compared with something or someone. It may be tempting to compare it to the pre or post-campaign period or with people who were not exposed to the advertisement. However, due to the high probability of seasonal and media bias, these types of comparisons should be avoided.

This is where the ability to target comes into effect. Users are initially split into two groups with a real ad shown to one group and a dummy ad shown to the other group. Aside from the difference in ads, both groups share the same qualities such as gender and age, the amount of time they are exposed to the ad, and the medium through which the ad is delivered. Thus, the only difference between the two groups is whether they were exposed to the real ad or the dummy ad. If a difference in brand awareness, favorability, or purchasing intent is seen between these two groups, we can safely assume the difference is the result of ad exposure.

Proper measurement to ensure proper learning and action

Despite the solutions mentioned above, problems related to measuring effective advertising still exist. For one, there’s still the issue of evaluating the role of online and offline advertising in a cross-media environment. Furthermore, methods to measure the effectiveness of ads that straddle multiple devices such as mobile devices and personal computers are not yet fully developed. In addition, the digital frontier continues to expand into social media as well as into other areas.

However, the most important part of "learning for future action" is the ability to accurately measure effectiveness. Decision making should be based on evidence rather than novelty, popularity, or a fixation on what was successful in the past. For that reason, we need to adopt a critical mindset which does not simply accept survey results. There needs to be courage to state the truth even if it is inconvenient. There needs to be an understanding of the technologies to solve issues, and the creativity for how to use them. The result of this continuously persistent effort? We will slowly be able to identify "which half of the advertising investment was a waste."

In Part Two, we’ll discuss the need for the agility of effectiveness verification and how digital technology can improve the feedback from complex marketing actions to adjust marketing policies in a timely manner.