Human Motion Analysis with Deep Metric Learning

TL;DR

This paper introduces a novel deep metric learning approach using attentive recurrent neural networks to better measure human motion similarity, outperforming traditional metrics in tasks like gait analysis and action retrieval.

Contribution

It presents a triplet-based deep metric learning framework with a new objective and architecture tailored for human motion data, handling variable input sizes and improving semantic separation.

Findings

Significant performance improvements over traditional metrics.

Effective handling of varying input sizes in motion data.

Enhanced semantic separation in motion embedding space.

Abstract

Effectively measuring the similarity between two human motions is necessary for several computer vision tasks such as gait analysis, person identi- fication and action retrieval. Nevertheless, we believe that traditional approaches such as L2 distance or Dynamic Time Warping based on hand-crafted local pose metrics fail to appropriately capture the semantic relationship across motions and, as such, are not suitable for being employed as metrics within these tasks. This work addresses this limitation by means of a triplet-based deep metric learning specifically tailored to deal with human motion data, in particular with the prob- lem of varying input size and computationally expensive hard negative mining due to motion pair alignment. Specifically, we propose (1) a novel metric learn- ing objective based on a triplet architecture and Maximum Mean Discrepancy; as well as, (2) a novel deep architecture based on attentive recurrent neural networks. One benefit of our objective function is that it enforces a better separation within the learned embedding space of the different motion categories by means of the associated distribution moments. At the same time, our attentive recurrent neural network allows processing varying input sizes to a fixed size of embedding while learning to focus on those motion parts that are semantically distinctive. Our ex- periments on two different datasets demonstrate significant improvements over conventional human motion metrics.