AI-Powered Augmented Reality as a Threat Vector for Human Manipulation

TL;DR

AI-powered augmented reality, when controlled by unregulated entities, poses significant risks of manipulation, propaganda, and disinformation, necessitating policy measures to mitigate potential harms.

Contribution

This paper highlights the potential dangers of AI-driven AR as a manipulation tool and proposes policy directions to address these emerging risks.

Findings

AI-powered AR can be used for targeted manipulation and influence.

Unregulated control of AR systems increases risks of propaganda and disinformation.

Policy measures are needed to mitigate manipulation risks.

Abstract

Augmented Reality (AR) is a powerful perceptual technology that can alter what users see, hear, feel, and experience throughout their daily lives. When combined with the speed and flexibility of context-aware generative AI, the power is greatly expanded, allowing individual users to be targeted with custom-generated AR experiences that are instantly tailored to who they are, where they are, and what they are doing. This can transform the physical world into a magical place, but only if the augmentation of a user's environment is enacted for their personal benefit and best interests. Instead, if AI-powered AR systems are controlled by unregulated third parties, such as large corporations or state actors, individually adaptive AR experiences could be deployed as a dangerous form of targeted influence. In fact, if the industry adopts an advertising business model for AI-powered AR devices, context-aware generative influence could become a widely used manipulative path for promotion of products and services in the physical world. Worse, similar techniques could be used for political influence, propaganda, and disinformation. This chapter reviews the power and flexibility of AI-generated augmented reality, explores the risks that emerge when used for persuasion, manipulation, or influence, and proposes policy directions to mitigate these risks.