Peeking into the Future: Predicting Future Person Activities and Locations in Videos

TL;DR

This paper introduces a multi-task learning system that predicts future pedestrian paths and activities in videos, demonstrating improved accuracy and providing insights into behavioral prediction.

Contribution

It presents the first joint model for predicting both future paths and activities, leveraging rich visual features and auxiliary training tasks.

Findings

State-of-the-art performance on trajectory prediction benchmarks

Joint modeling improves future path prediction accuracy

Capable of meaningful future activity prediction

Abstract

Deciphering human behaviors to predict their future paths/trajectories and what they would do from videos is important in many applications. Motivated by this idea, this paper studies predicting a pedestrian's future path jointly with future activities. We propose an end-to-end, multi-task learning system utilizing rich visual features about human behavioral information and interaction with their surroundings. To facilitate the training, the network is learned with an auxiliary task of predicting future location in which the activity will happen. Experimental results demonstrate our state-of-the-art performance over two public benchmarks on future trajectory prediction. Moreover, our method is able to produce meaningful future activity prediction in addition to the path. The result provides the first empirical evidence that joint modeling of paths and activities benefits future path prediction.