VISTA: A Vision and Intent-Aware Social Attention Framework for Multi-Agent Trajectory Prediction

TL;DR

VISTA is a novel transformer-based framework that jointly models agents' goals and social interactions for accurate and safe multi-agent trajectory prediction in dense environments.

Contribution

It introduces a recursive goal-conditioned transformer with social-token attention and interpretable social influence maps, advancing multi-agent forecasting methods.

Findings

Achieves state-of-the-art accuracy on MADRAS and SDD benchmarks.

Reduces collision rates from 2.14% to 0.03% on MADRAS.

Attains zero collisions on SDD while improving trajectory metrics.

Abstract

Multi-agent trajectory prediction is crucial for autonomous systems operating in dense, interactive environments. Existing methods often fail to jointly capture agents' long-term goals and their fine-grained social interactions, which leads to unrealistic multi-agent futures. We propose VISTA, a recursive goal-conditioned transformer for multi-agent trajectory forecasting. VISTA combines (i) a cross-attention fusion module that integrates long-horizon intent with past motion, (ii) a social-token attention mechanism for flexible interaction modeling across agents, and (iii) pairwise attention maps that make social influence patterns interpretable at inference time. Our model turns single-agent goal-conditioned prediction into a coherent multi-agent forecasting framework. Beyond standard displacement metrics, we evaluate trajectory collision rates as a measure of joint realism. On the high-density MADRAS benchmark and on SDD, VISTA achieves state-of-the-art accuracy and substantially fewer collisions. On MADRAS, it reduces the average collision rate of strong baselines from 2.14 to 0.03 percent, and on SDD it attains zero collisions while improving ADE, FDE, and minFDE. These results show that VISTA generates socially compliant, goal-aware, and interpretable trajectories, making it promising for safety-critical autonomous systems.