Deception at Scale: Deceptive Designs in 1K LLM-Generated Ecommerce Components

TL;DR

This study analyzes the prevalence of deceptive designs in 1,296 LLM-generated ecommerce components, revealing significant variation across models and effective strategies to reduce such designs.

Contribution

It provides the first large-scale analysis of deceptive design occurrence in LLM-generated web components and evaluates prompting strategies to mitigate this issue.

Findings

55.8% of components contained deceptive designs

DeepSeek-V3 produced the fewest deceptive components

Values-centered prompts effectively reduced deceptive designs

Abstract

Recent work has shown that front-end code generated by Large Language Models (LLMs) can embed deceptive designs. To assess the magnitude of this problem, identify the factors that influence deceptive design production, and test strategies for reducing deceptive designs, we carried out two studies which generated and analyzed 1,296 LLM-generated web components, along with a design rationale for each. The first study tested four LLMs for 15 common ecommerce components. Overall 55.8% of components contained at least one deceptive design, and 30.6% contained two or more. Occurence varied significantly across models, with DeepSeek-V3 producing the fewest. Interface interference emerged as the dominant strategy, using color psychology to influence actions and hiding essential information. The first study found that prompts emphasizing business interests (e.g., increasing sales) significantly increased deceptive designs, so a second study tested a variety of prompting strategies to reduce their frequency, finding a values-centered approach the most effective. Our findings highlight risks in using LLMs for coding and offer recommendations for LLM developers and providers.