Death by GPS
“Death by GPS describes what happens when travellers, unfamiliar with the area, follow the instructions and not their senses”
In James Bridle’s New Dark Age there are a series of anecdotes about the dangers of Automation bias. These stories range from Pilots shutting down engines unneccesarily, to tourists driving headlong into lakes or off into the middle of Death Valley before running out of petrol and perhaps more importantly, water. There is no malice here and in the case of the latter incidents, “The GPS signal wasn’t spoofed…the computer was simply asked a question, and it answered - and humans followed that answer to their death” (Bridle, p.43, 2018).
Human beings are lazy - and when possible, we like to delegate intensive activities (like complicated calculation or judgment) to someone, or something else. “Automation bias refers to our tendency to favor suggestions from automated decision-making systems and to ignore contradictory information made without automation, even if it is correct” (Hoffman, 2024).
In the insight space, its been interesting to see the rise of ‘synthetic sample’ within consumer research. Interesting because of the prevalence of skeuomorphic components to the marketing of this research methodology (referencing sample volumes by denoting n=x, detailing the demographic breakdown of samples and how they’ve been constructed for example) but also interesting because of the way the outputs derived from synthetic sample are already being treated with such credence. Early exposure to this work seems at best unusual and at worst like utter nonsense.
One recent project that I have seen the output from suggested the existence of an ‘evening indulgence moment’ for a coffee brand in the UK. Not only this, but the study suggested it represented a significant volume and value opportunity for the brand in question. When challenged, the response was that the LLM had spoken. To me, this stinks. And seems utterly counter intuitive to my experience of the category. There is a reason that most cafès shut mid afternoon. Caffiene and afternoons don’t typically mix unless you’re working on a pitch.
Use of synthetic sample has clear benefits vs. traditional methods of recruiting sample - not least the speed, flexibility and costs involved vs. more established methods. However, stuff like this is dangerous. In some circles, there is a danger that research is already underfunded and seen as low priority. Adoption of LLM based research and synthetic respondents may seem like a way of overcomming these barriers, but in reality the output will not do the discipline of research or it’s practioners any favours.
As Nik Samoylov of Conjointly says so aptly, “Every industry or field of knowledge has its own faux alter-ego. Astronomy has astrology. Medicine and homoeopathy. The time has finally come for market research to have its own. Its name is synthetic respondents.” (Samoylov, 2024). As Samoylov suggests in an article on the topic, the data used to train the different LLMs that are used by consumers and companies alike contains bias. It might be that in some markets around the world people do match the behaviours outlined in the study listed above. The challenge is whether or not the markets you’re interested in are the same...
Generative AI is already beginning to meaningfully change our work. It will provide us with new routes to information. New routes to ideas. And new methods of working. However, these benefits are not without danger. The danger of Automation bias is only going to increase as we build more automation into our personal and professional lives. Generative AI’s increased prevelance within many types of work within advertising will require the ‘hands on keyboards’ to be not just vigilant of inaccuracy within the information, but to think critically about the information too and avoid ‘death by GPS’.
Bibliography
Bridle, J. (2018). New Dark Age: Technology and the End of the Future. London: Verso.
Hoffman, B. (2024, March 10). Automation Bias: What It Is And How To Overcome It. Retrieved May 2024, from Forbes: https://www.forbes.com/sites/brycehoffman/2024/03/10/automation-bias-what-it-is-and-how-to-overcome-it
Samoylov, N. (2024, March 11). Synthetic Respondents are the Homoeopathy of Market Research. Retrieved May 2024, from Conjointly: https://conjointly.com/blog/synthetic-respondents-are-the-homeopathy-of-market-research/
