Bongers, M. & Hofmeyr, J. 2010. ‘Why modeling averages is not good enough – a critique of Double Jeopardy.’ Journal of Advertising research, 50:3, 323-33.

A longer explanation of the mistakes made in the above article can be read in:

Sharp, B., Wright, M., Dawes, J., Driesener, C., Meyer-Waarden, L., Stocchi, L. & Stern, P. 2012. ‘It’s a Dirichlet World: Modeling individuals’ Loyalties reveals How Brands Compete, Grow, and Decline.’ Journal of Advertising Research, 52:2, 203-13.

Here is a short critique:

The title is misleading, this is not a critique of the empirical phenomenon ‘double jeopardy’ but of the theoretical model ‘the Dirichlet’ – a stochastic model of purchase incidence and brand choice, which predicts double jeopardy and several other empirical laws in choice behaviour (see Ehrenberg et al 1994). The critique is naive, and the “test” of the Dirichlet is wrong.

To explain:

The Dirichlet is used daily within the marketing science units of corporations to benchmark their brand metrics against the model’s predictions for patterns of buying in a stationary and non-partitioned market. It is useful because the market conditions it models are well understood. It is also interesting to marketing theory because these conditions have been shown to be so prevalent (contrary to the world portrayed in most marketing textbooks).

Bongers & Hofmeyr use the purchasing of non-stationary brands to test an assumption of a stationary model. Even if they had found something real all they’d be saying was that non-stationary behaviour doesn’t look stationary.  Unsurprising.

It is, however, good and appropriate to question the underlying assumptions of models – even ones that work very well. In this case there has been more than 30 years of serious investigation of the NBD and Dirichet’s underlying assumptions (e.g. Kahn and Morrison 1989), a literature that the authors of this paper should have read. There is also new work seeking to expand such models to non-stationary conditions and to add co-variates (causal variables).

The Dirichlet belongs is a class of stochastic models, which is a technical way of saying that it assumes a particular distribution of purchase probabilities (concerning the probability to buy from the category, and the probability to buy particular brands within the category). These purchase probabilities (loyalties) for each buyer in the population are fixed in the model, that’s why we say it’s stationary – and therefore the brands obviously don’t change share because if people aren’t changing their loyalties then brand shares stay stable (which in reality they often do, at least over normal planning periods).  These stationary benchmarks are useful to compare change, when it happens, against.

Bongers & Hofmeyr tackle the Dirichlet model’s assumption that consumers have steady-purchase probabilities (steady loyalties); their paper attempts to refute this by showing a selection of purchases of individual panellists of non-stationary brands. They see what looks like lots of variation, natural wobble in purchase runs, and mistakenly interpret this as changes in loyalties.  The Dirichlet correctly incorporates a degree of random variation in purchasing even for stable loyalties.  Now if I gambled regularly I’d have a steady on-going propensity to lose money at the casino – but some nights I would actually make money yet that doesn’t mean the casino investors assumptions are wrong.  All B&H show is that runs of purchases exhibit variation (as do gamblers), rather than nice neat identical purchase weights in each quarter period. Similarly, if we had a panel of coin tossers we would see that only a few panel members made nice neat runs of tosses HTHTHTHT. If we looked at small runs of tosses we would see very many where ‘Heads” was far from 50% of the tosses. However, it would be foolish to send off a paper to a statistical journal critiquing the long-standing assumption of coins being weighted 50:50.

Now the Dirichlet models something much more complex than coin tossing. We have the probability to buy from the category mixed with the probability to buy particular brands.

The Dirichlet assumes steady-state probabilities with substantial stochastic wobble (around each individual’s steady-state mean). The multinomial assumption of choices amongst available brands means that a buyer with a 10% probability of buying the brand will buy it in the long run on 10% of their category purchase occasions but this buying will be in an as-if random fashion independently of the brand they bought on last occasion. Put simply when you look at any individual’s brand purchasing for short periods you see a lot of stochastic variation (which the model accounts for – hence its very accurate predictions).

It’s quite reasonable that even though I didn’t buy chocolate last quarter that I still have a on-going probability of buying it 4 times a year on average – that doesn’t mean I buy it exactly 4 times every year, some years I only once or twice, some years 8 times (and how I distribute my purchases amongst brands adds further (predictable stochastic) variation).

The authors don’t understand this stochastic variation and it has tripped them up. If it is any consolation it’s not uncommon for analysts to misunderstand stochastic variation in purchase data. Common mistakes include taking a group of heavy buyers (e.g. the top 20%) then noticing that in a subsequent period their purchasing is lighter and assuming that real changes in propensity have occurred (rather than regression to the mean). Or simply noticing that a person who bought in one period did not in a subsequent period and inferring that they have defected from the brand. I recommend reading Schmittlein, Cooper and Morrison (1993) for their discussion on true underlying propensities.

References:

Ehrenberg, Andrew S C, Mark D Uncles, and Gerald G Goodhardt (2004), “Understanding brand performance measures: using Dirichlet benchmarks,” Journal of Business Research, 57 (12), 1307-25.

Kahn, Barbara E. and Donald G. Morrison (1989), “A Note on ‘Random’ Purchasing: Additional Insights from Dunn, Reader and Wrigley,” Applied Statistician, 38 (1), 111-14.

Schmittlein, David C., Lee G. Cooper, and Donald G. Morrison (1993), “Truth in Concentration in the Land of (80/20) Laws,” Marketing Science, 12 (2), 167-83.