I have just read the updated summary of findings on Attention and I am keen to have more details on it. Is there a more detailed report that I can have access to? I’d like to better understand the methodology and see actual numbers behind this conclusion. It’s going against the “trend” which is not a bad thing but requires even more data point if that’s the position we should have on the topic.
Thank you for reaching out about this Attention paper. This is indeed a complex topic, and we absolutely appreciate that a lot of what we say here runs counter to current claims, particularly of proponents of Attention measurement in industry.
But in the meantime, a couple of considerations for you:
Firstly, as an independent university-based research Institute, our job is to critique such trends in industry, and call out claims that are not justified by the data where we find them. Sampling theory is of course crucial in determining the validity of individual studies, but we place a lot more emphasis on building generalised evidence across a range of conditions (studies, researchers, markets, metrics, etc). As you note, the more data points, the better! When we review a topic (such as attention in this instance) we aim to consider as many studies, data sets, journal articles, industry claims, points of view, etc on the topic as possible in order to draw our conclusions. The scientific philosophy that we bring to the research means we are less enamoured by a single study (even with a massive sample size, or great R2). For many reasons, such studies can be anomalies, and we respect our sponsors too much to have them jumping at shadows, so we want to make sure that we are very confident in what we are sending out.
With this in mind, we have drawn on a range of materials to arrive at this white paper on Attention. For instance, as you noted we refer to the ARF study. This too is a meta analysis of 32 different studies. One of the important conclusions, the authors draw is the great elephant in the room amongst proponents of attention measurement – that is, few (actually none that I know of) can prove a causal link between paying attention to ads and a sales response. The ARF has had a working committee looking into this for some time, (I was actually a member a couple of years ago). This piece draws on many (32) data sets that were made available to the ARF across many providers, using different metrics, etc.
We also draw on a broader body of knowledge from blind reviewed journal articles and published studies going back over 70 years for our white paper. Key to this literature is the timeless fact that low attention processing is (and remains) normal, and is the context in which ads are experienced. As we acknowledge, some attention is necessary for an effect, but beyond this there are not levels of attention (i.e. active being better than passive, etc). This levelled interpretation is necessary for media to justify that the currency should be based on attention. Certainly you can code for more or less attention in most attention data, but there is little evidence that the level of attention matters for advertising effectiveness.
We also have multiple data sets that we have examined to draw the conclusions in our white paper. These studies use a range of different operationalisations of the Attention construct. These studies vary from a large data set (~70,000 ads) from a global proprietor of eye-gaze attention data, to, multiple smaller studies, that operationalise attention by way of biometric and neuro (and others in between). From this range of studies, we have been able to identify that the way you measure “Attention” makes a big difference (also what the ARF concluded). For instance, eye gaze (as the most commonly available product on the market), is not a less sensitive measure of the same underlying construct, but rather it measures something different from the processing that neuro measures capture. It is often claimed by those selling gaze, that it is good enough if you can’t afford neuro, and that it is a more realistic measure than lab settings which are the typical setting for neuro measurement. However, the evidence suggests that gaze is not measuring processing, at least not like neuro can. This difference in what is being measured is reflected in the contradictory results that have been observed across attention studies that use different measures.
We also note that there is a push by proponents of attention measurement for attention to be incorporated in the media currency. Aside from the clear vested interest these parties have in pushing for its inclusion in the currency, we can see clearly in the data that attention is a concept that is brought to the advertising environment by the audience (that is, individuals pay more attention if they are older, are less educated, are from certain cultural backgrounds, at given times of the day). It is the audience’s circumstance that determines the attention they will pay to an ad, not the media in which that ad is placed. The media doesn’t control the attention, making it difficult for them to justify charging based on it.
E.R
19 August 2024
Link: https://sponsors.marketingscience.info/frequently-asked-questions/can-you-provide-more-details-on-attention/
Copy to Clipboard