Learning event representation: As sparse as possible, but not sparser

TL;DR

This paper explores the use of qualitative spatial reasoning features for classifying human-object interactions in 3D space, demonstrating significant performance improvements over quantitative features and proposing a simple, visualizable representation method.

Contribution

It introduces a modified QSRLib for feature extraction in 3D activity classification, showing qualitative features outperform quantitative ones and enabling a visualizable 2D action composition approach.

Findings

Qualitative spatial features improve classification accuracy.

Sequential QSR features yield the best performance.

Proposed method allows simple visualization of 3D activities.

Abstract

Selecting an optimal event representation is essential for event classification in real world contexts. In this paper, we investigate the application of qualitative spatial reasoning (QSR) frameworks for classification of human-object interaction in three dimensional space, in comparison with the use of quantitative feature extraction approaches for the same purpose. In particular, we modify QSRLib, a library that allows computation of Qualitative Spatial Relations and Calculi, and employ it for feature extraction, before inputting features into our neural network models. Using an experimental setup involving motion captures of human-object interaction as three dimensional inputs, we observe that the use of qualitative spatial features significantly improves the performance of our machine learning algorithm against our baseline, while quantitative features of similar kinds fail to deliver similar improvement. We also observe that sequential representations of QSR features yield the best classification performance. A result of our learning method is a simple approach to the qualitative representation of 3D activities as compositions of 2D actions that can be visualized and learned using 2-dimensional QSR.