# TrajFusionNet: Pedestrian Crossing Intention Prediction via Fusion of Sequential and Visual Trajectory Representations

## TL;DR

TrajFusionNet is a transformer-based model that predicts pedestrian crossing intentions by fusing sequential trajectory data and visual scene information, achieving state-of-the-art accuracy efficiently.

## Contribution

It introduces a dual-branch transformer model combining trajectory and visual data for improved crossing intention prediction.

## Key findings

- Achieves the lowest inference time among current methods.
- Sets new state-of-the-art results on three pedestrian crossing datasets.
- Effectively combines trajectory and visual cues for accurate predictions.

## Abstract

With the introduction of vehicles with autonomous capabilities on public roads, predicting pedestrian crossing intention has emerged as an active area of research. The task of predicting pedestrian crossing intention involves determining whether pedestrians in the scene are likely to cross the road or not. In this work, we propose TrajFusionNet, a novel transformer-based model that combines future pedestrian trajectory and vehicle speed predictions as priors for predicting crossing intention. TrajFusionNet comprises two branches: a Sequence Attention Module (SAM) and a Visual Attention Module (VAM). The SAM branch learns from a sequential representation of the observed and predicted pedestrian trajectory and vehicle speed. Complementarily, the VAM branch enables learning from a visual representation of the predicted pedestrian trajectory by overlaying predicted pedestrian bounding boxes onto scene images. By utilizing a small number of lightweight modalities, TrajFusionNet achieves the lowest total inference time (including model runtime and data preprocessing) among current state-of-the-art approaches. In terms of performance, it achieves state-of-the-art results across the three most commonly used datasets for pedestrian crossing intention prediction.