Position: Assistive Agents Need Accessibility Alignment

TL;DR

This paper emphasizes the importance of accessibility alignment in assistive agents for BVI users, proposing a lifecycle-oriented design pipeline to address current mismatches and improve inclusivity.

Contribution

It introduces the concept of accessibility alignment as a core design objective and proposes a comprehensive lifecycle approach for developing inclusive assistive agents.

Findings

Analysis of 778 assistance task instances reveals systematic failures in current AI assistive systems.

Current agentic AI often assumes sighted interaction, leading to mismatches with BVI user needs.

A lifecycle-oriented design pipeline is proposed to improve accessibility alignment in assistive agents.

Abstract

Assistive agents for Blind and Visually Impaired (BVI) users require accessibility alignment as a first-class design objective. Despite rapid progress in agentic AI, most systems are designed and evaluated under assumptions of sighted interaction, low-cost verification, and tolerable trial-and-error, leading to systematic failures in assistive scenarios that cannot be resolved by model scaling or post-hoc interface adaptations alone. Drawing on an analysis of 778 assistance task instances from prior work, we show that current agentic AI remain prone to failure in assistive scenarios due to mismatches between sighted-user design assumptions and the verification, risk, and interaction constraints faced by BVI users. We argue that accessibility should be treated as an alignment problem rather than a peripheral usability concern. To this end, we introduce accessibility alignment and propose a lifecycle-oriented design pipeline for accessibility-aligned assistive agents, spanning user research, system design, deployment and post-deployment iteration. We conclude that BVI-centered assistive tasks provide a critical stress test for agentic AI and motivate a broader shift toward inclusive agent design.