The public, with no small help from the media sniffing a great story, is ready to believe in the supernatural powers of a mostly unproven targeting strategy.

— Martinez (2018) “The Noisy Fallacies of Psychographic Targeting”, Wired.

Background

The Ehrenberg-Bass Institute has compiled a great deal of different evidence that all points in the same direction… that for growth or maintenance a brand needs to reach all category buyers. “Target the market” as we say. This isn’t the same as (mythical) textbook mass marketing because reaching the whole market nearly always means some tailoring to some important differences between customers. That’s why we use the term sophisticated mass marketing: tailoring advertising and messaging where it is valuable to do so, based on a sound understanding of differences in consumer behaviour.

Mass customisation could be smart targeting, i.e. a way to reach the entire market, but with many different messages. New technologies offer the prospect of sending different advertising messages to different consumers with the promise of far lower costs of customisation than that which previously existed (e.g. making a different TV commercial for every state in the USA).

Plus, new technologies offer the ability to tailor these messages not just based on consumer demographics, but on their past behaviours and psychographics. Marketers have had a long infatuation with psychographics; commissioning many psychographic segmentation studies, that while entertaining turned out to be of little practical value. One reason being the difficulty (or impossibility) of buying advertising media on psychographic grounds.

For some while now social media has offered advertisers the ability to target messages based on the demographic information that users of the social media provide about themselves (e.g. where they live, their gender, their age). More recently, some junior academics have claimed that much more can be known (and exploited) about social media users by examining their “likes”, as proxies for their psychographics. In 2013 Kosinski and colleagues published a paper titled “Private traits and attributes are predictable from digital records of human behavior”:

“We show that easily accessible digital records of behavior, Facebook Likes, can be used to automatically and accurately predict a range of highly sensitive personal attributes including: sexual orientation, ethnicity, religious and political views, personality traits, intelligence, happiness, use of addictive substances, parental separation, age, and gender.”

In reality the article shows good-to-modest correlations between certain patterns of Facebook “likes” and personal attributes. Age, gender and ethnicity were most accurately estimated. Some correlations, such as the link between high intelligence and liking Curly Fries, are likely to be spurious (i.e. won’t generalise to another data set). While the authors use the term “prediction” they didn’t test predictions, their study merely looked for correlations within a single dataset. Any generalisations are therefore likely to be far less accurate when applied to any other/future data. As our colleague John Dawes and others have shown – model fitting is easy, prediction is difficult.

Today Michael Kosinski and David Stillwell are (in)famous for this work because it provided the inspiration for their Cambridge University colleague Aleksandr Kogan to develop an app that ironically bypassed the need to infer user characteristics by simply going straight to harvesting the personal data that millions had already independently revealed to Facebook in their user profiles. These data were sold to Cambridge Analytica, a company that allegedly used it for mass customisation of political advertising (for Ted Cruz against Donald Trump, and unsuccessfully).

Evidence on psychological targeting

What matters to us though is that Kosinski and Stillwell (together with colleagues) also published work on the effectiveness of psychologically-tailored advertising messages. Their main experiment delivered 3.1 million exposures of advertisements for a single UK beauty retailer resulting in 10,000 clicks and 390 sales. Ten different creative executions were delivered in a 2×2 between subjects experiment, i.e. some introverted subjects were delivered advertisements tailored for introverts (a congruent condition) and some were delivered advertisements tailored for extroverts (an incongruent condition); the design was mirrored for extroverted consumers.

It’s important to note then that this was not a test against a control of simply sending one message designed to have universal appeal. But rather, the comparison here is between a creative message designed to match to the psychological profile against a creative message crafted to be completely incongruent i.e. an advertisement for extraverts sent to introverts and vice versa. So, it’s a weak test and one that the congruent ads should have easily won – but they didn’t.

Firstly, we note that the click-through rates (CTRs) were low, a bit lower than is considered “industry average” for Facebook ads. Introvert-styled ads received an average CTR of just 0.33% while extravert-styled ads received 0.34%. Perhaps this is because they were developed by academics rather than advertising professionals?

Low CTR occurred even for the ads that were tailored to the personality of the viewers. Note that the first row should feature the higher click-through rates because these are when the advertisements matched to (‘congruent’ with) the viewer’s personality. 

The data is orientated to focus on the effects of congruency within each ad-style treatment (comparing down in columns):

Note: Data from appendix table S5.

The top row figures, where the ad message was congruent to the psychological profile of the recipients, should be [much] higher than the lower row. However, this is not the case. More importantly, the relationship is inconsistent.

This difference in response between congruency states within ad styles (though with contradictory reverse effects between groups that cancelled each other out in aggregate) is actually solely the result of there being a difference in CTR between personality target groups and no difference in CTR between congruency states. The introverted group always clicked a bit more, even to the ads that were crafted for extraverted people; in fact ever so slightly more for ads that were incongruent i.e. crafted to appeal to extraverts.

Here the data have been re-orientated to focus on the effects of congruency within each personality group (again, comparing down in columns):

There were some differences in CTRs among the different creative executions. A very inconvenient finding is that the ad to the Introverted target group that yielded the highest response was incongruent, while similarly, the ad to the extraverted target group with the lowest ad response was a congruent extraverted ad style.

It took some heroic analysis, dare we say sophistry, for the authors to spin these results into a positive story for psychologically-tailored advertising.

The authors highlight the effects of psychological targeting on behaviour beyond click-through, what happened after click-through was no longer under their experimental control. The null results for the primary behaviour they did influence, click-through, mirror a long history of research showing there is little relationship between what people think and what they do. A recent example is the Implicit Association Test, which measures a person’s potentially unconscious biases to explain and help them alter their explicit behaviour. A recent meta-analysis, which examined 499 studies over 20 years involving 80,859 participants, concluded that there is very little evidence that changes in implicit bias have any relationship with changes in a person’s behaviour. Part of the reason is the low test-retest reliability of the IAT. But even people aware of their implicit biases have the ability to avoid acting on them, in response to many other internal and external influences. An introvert targeted by an introverted message may like that message more than an extroverted message, and it may even change their mind. But whether that specific change of thinking leads to a change of behaviour is unlikely and is not strongly supported by the Matz et al. study.

The lesson

Last year Mark Ritson predicted an electoral landslide for the UK Conservative Party based on their clever Facebook micro-targeting of customised messages to individuals. Given that the opinion polls strongly supported a Conservative win it didn’t seem a bold prediction… but the election results shocked many, including Mark, with the Conservatives barely retaining power.

All this reminds us that the assumption that mass customisation is more effective than mass appeal is speculation; and is certainly not necessarily the case. The evidence is not supportive, even though people have tried to find it. So the lesson is to be wary, to not assume, but rather to do tests, because the effectiveness of ad copy does vary enormously.

Today, already, most media offer advertisers the ability to target geographically, and for many brands this is all the targeting they need – to match mental availability to physical availability and demand. Most media also offer advertisers the ability to target by time, gender, age and interests such as cooking or sports. For many brands, the incremental gains that might come from any other tailoring would be very small indeed. So Facebook is a vast medium which is of great value to advertisers irrespective of whether it offers psychographic targeting.

No one should spend large amounts of their company’s money on systems to design micro-segments and deliver mass customised advertising, not without first doing a great deal of testing – preferably by people not strongly interested in a positive result. There is much to learn, we need to know under what conditions customisation actually yields greater overall response and when any extra value of customisation exceeds the cost of the customisation.

All advertisers should be wary of firms like Cambridge Analytica selling tall stories and unsubstantiated claims about advertising effects.

For more guidance on segmentation and targeting please see our Marketing: Theory, Evidence, Practice textbook chapter six.

Footnote

We report our analysis of the first study reported in Matz et al. 2017 because it is the largest experiment and directly based on psychological assessment of the participants. We focussed, naturally, on the effect of ‘congruence’ of the psychological targeting on the behavioural response (CTR). Because that was the purpose of the tailored advertising, to make viewers click-through to the website (i.e. the store). Matz et al. ignored this behavioural response and instead examined later purchases from the website (conversion rate) – they give no logic for this, nor give any explanation for why it might be expected that later conversions would vary with prior advertising congruence. That there was some (small) variation suggests some sort of selection effect.

What about study 2?

Didn’t the psychological testing result in 40% higher response rate?

Yes, unlike in Study 1 where there was no difference in behavioural response between congruency states, in study 2 congruent ads did result in a 40% higher CTR (on average) than did incongruent ads. However, this is mostly due to the higher CTR for congruent ads (50% higher) within only the LO ad style. Additionally, and disturbingly for Matz et al.’s thesis is that the measure of overall campaign efficiency, Conversions per Impression (or, Reach) or what Matz et al. erroneously call Conversion Rate is higher for incongruent ads in the LO ad style than for even congruent ads in the HO ad style.

A shorter version of this report was published in the academic literature, in the Proceedings of the National Academy of Science:
Sharp, B., N. Danenberg and S. Bellman (2018). “Psychological targeting.” Proceedings of the National Academy of Sciences, August 21, 2018. 115 (34) E7890