Functional Misalignment in Human-AI Interactions on Digital Platforms

TL;DR

This paper discusses how social media algorithms optimize for predictable user behavior, leading to societal issues like polarization and mental health problems due to structural misalignment with human goals.

Contribution

It introduces the concept of functional misalignment, identifying three mechanisms causing societal harm from predictive algorithms, and proposes a research agenda to address this.

Findings

Algorithms optimize for reactive signals over reflective judgment.

Feedback loops amplify behavioral effects at scale.

Predictive accuracy can lead to societal adverse outcomes.

Abstract

Algorithmic systems, particularly social media recommenders, have achieved remarkable success in predicting behavior. By optimizing for observable signals such as clicks, views, and engagement, these systems effectively capture user attention and guide interaction. Yet their widespread adoption has coincided with troubling outcomes, including rising mental health concerns, increasing polarization, and erosion of trust. This paper argues that these effects are consequences of a structural functional misalignment between what algorithms optimize - predictable behavior - and the human goals these predictions are intended to serve. We propose that this misalignment arises through three mechanisms: (1) a bias toward modeling fast, reactive behavioral signals over reflective judgment, (2) feedback loops that couple user behavior with algorithmic learning, and (3) emergent collective dynamics that amplify these effects at scale. Together, these mechanisms explain how accurate individual-level predictions can produce adverse societal outcomes. We present functional misalignment as a unifying framework and outline a research agenda for studying and mitigating its effects in human-AI interaction systems.