​​If the purpose of your advertising and PR is to encourage… ​​
… visits to your website
​​… visits to your dealerships/branches/stores
​​… calls to your call centre
​​… online search for your brand

​​…then you should use such metrics to evaluate your advertising. Customer search metrics like the above, not only allow you to see how well your advertising is stimulating search but also allows you to see how well you are doing in converting enquiries into sales.

​​This has been standard practice for many years for any professional marketing department. So I hardly need write about it… but lately it’s been suggested that such metrics might have other uses. In particular, “share-of-search” has been touted as something of a miracle metric: cheap and predictive (but of what?).

​​In the words of famous scientist Carl Sagan “extraordinary claims require extraordinary evidence” (he also encouraged scientists to be open-minded “but not so open-minded that your brains fall out”). So let’s have a look at the claims for share-of-search, and some evidence.

Firstly, Google kindly make share-of-search data freely available, so its only cost is doing some data cleaning, calculating shares, and such like. So what of the second claim that search data can be used to make predictions? The key question is….

What sort of prediction would be useful?

There is no shortage of market research agencies offering special (and proprietary) ‘predictive’ metrics. To prove their claim of ‘predictive power’ they usually show that bigger brands (with more sales/share) score higher. But bigger brands, with more users, score higher on practically any positive attitudinal questions, claimed usage, advertising effectiveness measure and so on – so it’s very easy to create a new metric that correlates with market share. Hardly impressive, and not very useful: a metric that tells you what you already know (i.e. if your brand is big or small). This is not well appreciated in the marketing community and hence brand trackers are infected with dozens of these questions.

Note: In the book How Brands Grow check out the “My Mum” law (chapter 5). Also see chapter 3 “Meaningful Marketing Metrics” in our textbook for University students.

A slightly more impressive way to claim predictive power is to show that the metric goes up when sales rise, and declines when sales fall. A close correspondence between search volume and sales can occur because search precedes sales, and because sales encourage search (eg for warranty details or instruction manuals). Or, both can be caused by something else (like changes in physical availability), for example, a car company launches new models, gets them into dealers, advertises a lot – and of course both sales and search go up. They’re both caused by the same thing: producer activity.

Just because the metric moves with sales/share, it does not reveal beforehand that the brand is about to change trajectory: it only tells you once you already see it in your sales figures. Not very useful.

More tantalisingly, it’s been claimed that share-of-search might (sometimes and sometimes not) forecast a change in sales trajectory before it happens. This is interesting because it’s plausible… well, it’s plausible for what economists call “search goods”, where people do some investigating some time before they buy. There are such products, like cars and mobile phones, where many consumers go online to find out about them (including where to buy them), whereas last century people read paper brochures and used ‘yellow pages’ directories. There are also products like insurance and electricity/gas supply that are very likely to be ordered online, so naturally people use search to find the ecommerce site to buy. So, in these cases, online search is going to precede purchase, but…it might precede it only by minutes, which means it cannot forecast far ahead.

Some charts make it look like share-of-search is reacting far earlier than market share but this is artificial. That’s because it’s due to the market share being calculated over a longer historic period (e.g. sales last month), when share-of-search is calculated on searches done yesterday. And so what looks like prediction is largely back-casting. This is a one example of modellers’ sleight of hand.

​​Similarly, when a car or mobile phone company announce a new model share-of-search for this product rises. Search is preceding sales, which haven’t even started yet. Then later as the company readies itself to replace that model they naturally pull-back on advertising. Share of search starts declining, but sales continue for a while as dealers work to run out the remaining stock. This gives an inflated impression that search is predicting the brand’s trajectory when really the trajectory was already known, indeed decided.

Technology companies often have a very good idea of the sales trajectory of their new model months (even years) before they launch it. Sales will climb after launch, plateau rather soon in the technology cycle and then seriously decline as they (and rivals) launch better updated technology. It’s not surprising that search follows a similar pattern, and it’s not very insightful. Search spikes on the announcement of the new model, and then meanders along until fading on the announcement of a new model.

So what looks predictive turns out to be rather unexciting, not of practical value. Now….What would really be of great value would be if share-of-search could tell how successful a launch was going to be. To be useful, a forecast of greater than expected market demand would need to be made well ahead of time, to give enough warning so we could confidently ramp up factory capacity or secure more distribution. Likewise we’d really like to know well in advance if the launch is going to be a disappointment.

So far no one has made this claim of predictive power, but it’s still worth considering because this is the sort of prediction that would be of great practical value.

Apple AirPods are an example of a launch where there was far more demand than expected, and Apple was unable to supply sufficient product for many months. Wikipedia reports that “within two years, they became Apple’s most popular accessory, turning into a critical success and viral sensation”. They launched in December 2016, and a 2nd generation launched March 2019. Here is AirPods’ Google search history, which, hmmm, seems to match the product’s availability and sales rather than predicting its success. Apple knew they had a hit on their hands when they quickly ran out of stock. Did online search data help predict this runaway hit? It appears not.

And yet this is exactly the sort of launch where we’d expect/hope share-of-search to work best.

Nintendo Wii U was the opposite of Airpods: a disappointing launch that resulted in the second- worst performing Nintendo console of all time behind only the Virtual Boy, which gave users red-and-black headaches. In comparison, the Nintendo Switch sold as many units in three months as the Wii U sold in five years. But, you’d never know this by looking at search data (nor that Switch is on track to sell more in total than the original Wii ever sold). The search peaks for Switch are about as big as the peaks for Wii U, yet Switch sold massively more units.

The Wii U was only introduced in 2012, despite Google reporting search spikes in 2006, alongside the Wii.

Currently, we have a really interesting comparison, as both Sony and Microsoft are launching new models of their leading games consoles. Search data has predicted that Xbox Series X will be a more successful launch than PlayStation 5. Both showed spikes in search after their features/price were released, and Xbox charts ahead of PlayStation 5 in both the spike in interest and the subsequent pre-launch search. Even more so if we added in search for Xbox Series S, a cheaper version that has also been announced.

Both the new PlayStation and Xbox are set for launch in November 2020, so we shall see. Production of the current model of Xbox ended in July, while Sony has not stopped production of PlayStation 4, and there have been rumours of production constraints for PlayStation 5 (denied by Sony). All of this is in Xbox’s favour, but historically PlayStation has been the market leader.

A great test of any theory is the ability to make predictions: this is a good chance to find out how good this crystal ball really is.

Meanwhile we do know that search incorrectly predicted the popular vote and outcome in the US election, and, in spite of initial academic enthusiasm, turned out to be unable to predict the annual flu outbreaks.

In summary, in spite of some recent enthusiastic claims share-of-search appears unable to make reliable predictions that would be of practical value. The Ehrenberg-Bass Institute is continuing to gathering a wide array of evidence to assess this tool.

2024 Update

And now, we get to see the results of our crystal-ball gazing. Share of search predicted that in the launch of two new products, Xbox Series X would beat Playstation 5, and when combined with Series S, do so easily. But what really happened? In this update, we will first look at the current predictions made by share-of-search, to see what it continues to predict. Then, we can look at sales data to determine if there is any relationship between the predictions and the real world. 

First, we’ll look at the raw “search term” results, so that we can compare with the initial prediction, then use Google’s clustering to determine if that increases the accuracy of the measure. These “search term” results show the initial data from above, in the first two spikes. Beyond that, we see a very large spike in November 2020 – when both the Xbox and Playstation were launched. We also see smaller series of spikes in November and December of both 2021 and 2022. Xbox is higher in each instance.

The prediction in the search data, that the Xbox would have higher market share, continues – search results suggest that overall, Microsoft should have been selling more consoles than Sony.

Google Trends data for search terms

However, Google has ways of grouping searches and when we change to using that analysis, the picture changes. We’ve also included searches for the third console of interest, Nintendo’s Switch. Here, the story changes: the spikes we saw in the initial data still exist, but are reduced in size for Xbox (which now has both S and X consoles combined into a single set of searches). We still see the peaks in November and December, but now Playstation appears to be peaking at twice as high as Xbox. 

The prediction appears to have changed significantly: not only is search data predicting a different winner, but it needed the consoles to have launched, and for Google to have improved their data collection, for this result to appear.

Google Trends data for Topics

But what does the real world look like? Playstations outsell Xboxes at about a ratio of 2:1, which is (roughly, if you’re being generous) reflected in the search data. So which prediction came true? 

Unfortunately, neither. Current best estimates for global console sales stand at 21m Xboxes, 42m Playstation 5s, and 130m Switches. This huge disparity in total sales doesn’t appear in the data, outside of a peak for Nintendo Switch in March 2020, and then generally being very slightly higher than either console from November 2021 onwards. The Switch is very close to being the best-selling console of all time – but you wouldn’t know it from the data. 

Also not featured in the search data is the huge seasonality present in the sales data. While there is clearly some present, it does not reflect the importance of Christmas for the sales of game consoles.

Perhaps this most promising use of search data, predicting pent-up-demand, but alas this also doesn’t seem to work. Rumours of Playstation shortages were true, and it wasn’t freely available to buy until August 2022. You can see this clearly in the sales data, where PS5 had a quiet December ’21, but began to grow in sales from roughly August ’22 to a new plateau of roughly 250k weekly units, more than double Xbox’s 100k. We don’t see an indication of this in search data, and if we try to convert the raw search figures to a share metric, interest in the PS5 goes down in the lead up to the console’s best Christmas yet. Imagine if you had relied on this data to determine manufacturing!

Next up: can we compare snapshots of the market shares of brands in consumer surveys with the reported market shares of brands from share-of-search?