Graphing the Future: Activity and Next Active Object Prediction using Graph-based Activity Representations

TL;DR

This paper introduces a graph-based method for predicting ongoing human-object interactions and future active objects in videos, achieving high accuracy without modeling detailed motion or contact points.

Contribution

It proposes a novel graph matching approach using Graph Edit distance for classifying interactions and forecasting next active objects in video sequences.

Findings

High accuracy in action prediction on MSR Daily Activities dataset

Effective forecasting of next active objects on CAD120 dataset

Demonstrates the potential of graph-based methods for future interaction prediction

Abstract

We present a novel approach for the visual prediction of human-object interactions in videos. Rather than forecasting the human and object motion or the future hand-object contact points, we aim at predicting (a)the class of the on-going human-object interaction and (b) the class(es) of the next active object(s) (NAOs), i.e., the object(s) that will be involved in the interaction in the near future as well as the time the interaction will occur. Graph matching relies on the efficient Graph Edit distance (GED) method. The experimental evaluation of the proposed approach was conducted using two well-established video datasets that contain human-object interactions, namely the MSR Daily Activities and the CAD120. High prediction accuracy was obtained for both action prediction and NAO forecasting.