# Social-aware trajectory prediction using goal-directed attention networks with egocentric vision

## TL;DR

This paper introduces a new model for predicting future movement paths of road users by combining vision data and social interactions into a unified framework.

## Contribution

The novel SGANet model integrates social interactions, temporal dynamics, and destination prediction into a unified multimodal trajectory prediction framework.

## Key findings

- SGANet outperforms previous benchmarks on both homogeneous and heterogeneous road user datasets.
- Socially-aware interaction weighting significantly improves prediction accuracy for ego-vehicle maneuvers.
- Each network component in SGANet is validated as effective through extensive experiments.

## Abstract

This study presents a novel social-goal attention networks (SGANet) model that employs a vision-based multi-stacked neural network framework to predict multiple future trajectories for both homogeneous and heterogeneous road users. Unlike existing methods that focus solely on one dataset type and treat social interactions, temporal dynamics, destination point, and uncertainty behaviors independently, SGANet integrates these components into a unified multimodal prediction framework. A graph attention network (GAT) captures socially-aware interaction correlation, a long short-term memory (LSTM) network encodes temporal dependencies, a goal-directed forecaster (GDF) estimates coarse future goals, and a conditional variational autoencoder (CVAE) generates multiple plausible trajectories, with multi-head attention (MHA) and feed-forward networks (FFN) refining the final multimodal trajectory prediction. Evaluations on homogeneous datasets (JAAD and PIE) and the heterogeneous TITAN dataset demonstrate that SGANet consistently outperforms previous benchmarks across varying prediction horizons. Extensive experiments highlight the critical role of socially-aware interaction weighting in capturing road users’ influence on ego-vehicle maneuvers while validating the effectiveness of each network component, thereby demonstrating the advantages of multi-stacked neural network integration for trajectory prediction. The dataset is available at https://usa.honda-ri.com/titan.