It’s long been said that one of the most important benefits of online display advertising is that it can be targeted better than most older media. This offers the potential to avoid ‘wastage’ (wastage being paying for ad exposures that reach people outside of the media target). The Ehrenberg-Bass Institute has cautioned that any ability to target should be used carefully – that advertisers should avoid narrow targets that exclude category buyers (or future category buyers). We’ve encouraged advertisers not to simply assume that a more targeted media buy is better than a less targeted one, but rather to base their decisions on evidence, to calculate the costs of reach.

Earlier this year, we published an article in the Proceedings of the National Academy of Science weighing in on the Cambridge Analytica Facebook data scandal that showed that the power of psychographic targeting is much over-stated.

This important research paper further highlights this problem, with the caution to not assume that the promised targeting is actually being delivered. As the evidence presented here shows, the ability of data brokers to deliver accurate targeting information is far from impressive.

• Digital profiling correctly identified a male consumer only half the time, while the age bracket of consumers (tested case: 25-34 years) was correctly classified only slightly better than random guessing.
• Our test of six programmatic buying services found that almost half of the advertising impressions we paid for did not reach the target audience. This is largely because of the inaccuracy of their consumer profile targeting data.
• Advertisers are advised to demand full transparency about purchased data segments and to rely on first-party data wherever possible.

Targeting advertising to specific audiences (e.g. women aged under 35 years old) depends on being able to accurately identify the audience. For online advertising, consumers’ online browsing records are used to create digital profiles of the consumers. Data brokers then sell these profiles to advertisers via programmatic buying of media. Some marketing managers have dedicated large portions of their advertising budgets campaigns to using such third-party segments. For example, the data-targeting market is estimated to be worth over $10bn in the US alone. [1]

However, little is known about the accuracy of the audience identification that is provided by buying platforms (Goldfarb and Tucker 2011). In this report we investigate the accuracy of audience identification, by focusing on the two most widely-used targeting attributes, simply age and gender. [2]

1. Can data brokers accurately determine characteristics of individual online users?
   We compare the age tier and gender suggested by data brokers with those reported by the individual research participants themselves (who were members of a research panel).
2. Can programmatic targeting eliminate ‘wastage’?
   We examine the wastage level of targeted campaigns using Nielsen DAR/ Facebook data.
3. Are media campaigns using programmatic buying technology and third-party data more cost efficient than non-targeted media campaigns?
   We discuss the benefit-to-cost ratio of employing audience targeting for online campaigns.

[1] Winterberry Group (2018). The State of Data 2017. Technical report.

[2] Lotame Research Report (2018). The New State of Audience Data: Accuracy Matters.

We first linked 33,986 user profiles (comprising UK, US, New Zealand and Australian users) of Pureprofile, a high-quality and certified panel provider, with the consumer profiles provided by 14 global data brokers.[3]
We cannot reveal the identities of the tested data brokers, but they are among the leading data aggregators worldwide.

[3] The user profiles were matched with data brokers using web cookie syncing for four weeks at the end of 2016.

This first test looked at how well individual user characteristics can be identified through the digital profiling techniques (usually a black box) applied by data brokers. We used two separate tests of targeting and classification: (1) the age bracket “25-34 years”; and (2) the gender of male.

We found high and significant variation in accuracy across the tested data brokers for both gender (χ²(13) = 1295.0, p<.001) and age (χ²(8) =1409.7, p<.001). The results of each testable vendor for the two demographic variables are summarised in Tables 1a and 1b, respectively. Gender accuracy ranges from 26% to 63% for our test sample, with an overall average of only 42%. Given that random guessing delivers accuracy of about 50%, (i.e. half the population is male) using using data brokers to assess online browsing profiles for gender appears on average to be pointless.

Table 1a: Accuracy of tested data brokers for gender (male)

Turning to the ability of the data brokers to correctly identify consumers’ age group, their accuracy ranged from 18% to 37% in our tests, with an average of 26%. Random luck should have given them 18% (according to Statista, the age bracket of 25-34 year-old people represents approximately 17-18% of the internet user population [4]). So while this is an improvement over random guessing, the very high rate of misclassification almost completely undermines the validity and usefulness of advertising targeting. Even with the brokers who performed better than chance there was an average of 70% misclassification.

Table 1b: Accuracy of tested data brokers for age (bracket: 25-34 years)

Furthermore, the two vendors with the largest, most robust sample sizes (33,000 and 11,000) both yielded results that were only in line with chance. It was only with vendors with much smaller samples where the results were above chance.

[4] Statista (2015, 2018). U.S./ Australia: internet user age distribution.

In a second test, we provided a campaign brief with fairly broad targeting criteria to the service teams of six programmatic buying platforms (so called DSPs or demand side platforms)[5]. The goal of the test campaign was to target as many male individuals aged between 25 and 54 years as possible without exceeding 100,000 ad impressions. To validate the accuracy of this live campaign, we relied on Nielsen DAR data, which uses both its panel and Facebook data to validate user characteristics.

The ideal was to deliver all the impressions inside of the target market, and with maximum reach (i.e. no extra frequency).  So the best a campaign could achieve was to deliver an advertising impression to 100,000 males aged 25-54 year old.

Unfortunately, no campaign got anywhere close to this objective.  In fact, on average, they only reached about half that number of target viewer ‘opportunities to see’.

All six programmatic platforms wasted some of the 100,000 impressions by serving ads to some people more than once.  People saw between 1% and 41% more impressions than we had specified in our brief (reminder: we asked to maximise reach for a fixed number of impressions, meaning the goal was to aim for a frequency of one ad per person).  But the main cause of reduced reach into our target audience was mis-classification of viewers (i.e. serving impressions outside of the target). The overall results of the automated audience delivery are rather disappointing, with an average of 41% of impressions being off target despite the use of algorithmic optimisation.

Table 2: Accuracy and frequency of ad delivery in our live campaign

[5] The Interactive Bureau of Advertising (IAB) defines a DSP as “a technology platform that provides centralised and aggregated media buying from multiple sources including ad exchanges, ad networks and sell side platforms, often leveraging real time bidding capabilities of these sources.”

To further assess the efficiency of the achieved audience delivery in our live test campaign (the second test), we can compare the increase in audience identification with the natural distribution of the two characteristics. Male internet users aged 25-54 make up about 27%[6] of online users, leading to a relative improvement of audience matches relative to randomly selecting impressions of about 123% (0.59 divided by 0.265) through the programmatic audience delivery.

While this is an improvement over random chance, this reduced ‘wastage’ needs to be set against the extra costs that advertisers would incur for using programmatic audience delivery services. Beales (2010) analyses CPM data of 12 ad networks and shows that audience targeting for online ads creates extra costs of about 168 % on average in comparison to random placements.[7] Given that leveraging audience delivery therefore costs advertisers about 168% more, but only delivers an 123% improvement in performance, this appears an unattractive benefit-to-cost ratio.

[6] Statista (2015). Australia: internet user age distribution.

[7] We believe the CPM differences in Beales (2010) are highly conservative. In the authors’ experience, placements without specified audiences in open exchanges benefit from much lower CPMs, even up to 1000%.

Third-party data brokers appear to overstate the targeting accuracy that they can deliver, even on basic characteristics like age and gender. For our chosen age and gender attributes, the accuracy was on average only around that delivered by random guessing. In other words, third-party data performed on average barely better than using no targeting information at all.

To the best of our knowledge, there are presently no industry standards or certifications in place for third-party data. Moreover, most vendors do not share the methods or rules they apply to create their pre-defined audiences. Therefore, we recommend that business decision-makers should rely on first-party (their own) data wherever possible. Alternatively, clients dependent on third-party data are well advised to request full transparency about the applied user classifications and to carry out their own validation efforts to minimise ad spend wastage.