Fake Friends and Sponsored Ads: The Risks of Advertising in Conversational Search

TL;DR

This paper critically examines the risks and ethical challenges of advertising in conversational search, highlighting potential harms, misuse of data, and the 'fake friend dilemma' as AI becomes more integrated into information seeking.

Contribution

It introduces the concept of the 'fake friend dilemma' and discusses potential future forms and risks of advertising in conversational AI systems.

Findings

Advertising may degrade search quality and user trust.

Conversational agents could exploit user trust for commercial gain.

Risks include misinformation and harm in sensitive topics.

Abstract

Digital commerce thrives on advertising, with many of the largest technology companies relying on it as a significant source of revenue. However, in the context of information-seeking behavior, such as search, advertising may degrade the user experience by lowering search quality, misusing user data for inappropriate personalization, potentially misleading individuals, or even leading them toward harm. These challenges remain significant as conversational search technologies, such as ChatGPT, become widespread. This paper critically examines the future of advertising in conversational search, utilizing several speculative examples to illustrate the potential risks posed to users who seek guidance on sensitive topics. Additionally, it provides an overview of the forms that advertising might take in this space and introduces the "fake friend dilemma," the idea that a conversational agent may exploit unaligned user trust to achieve other objectives. This study presents a provocative discussion on the future of online advertising in the space of conversational search and ends with a call to action.