Temporal Rate Reduction Clustering for Human Motion Segmentation

TL;DR

This paper introduces a novel clustering method called Temporal Rate Reduction Clustering ($\text{TR}^2\text{C}$) for human motion segmentation, which learns temporally consistent representations aligned with a Union-of-Subspaces, outperforming existing methods.

Contribution

The paper proposes $\text{TR}^2\text{C}$, a new approach that jointly learns structured, temporally consistent representations for improved human motion segmentation.

Findings

Achieves state-of-the-art results on five benchmark datasets.

Effectively handles complex motions with cluttered backgrounds.

Outperforms existing subspace clustering methods.

Abstract

Human Motion Segmentation (HMS), which aims to partition videos into non-overlapping human motions, has attracted increasing research attention recently. Existing approaches for HMS are mainly dominated by subspace clustering methods, which are grounded on the assumption that high-dimensional temporal data align with a Union-of-Subspaces (UoS) distribution. However, the frames in video capturing complex human motions with cluttered backgrounds may not align well with the UoS distribution. In this paper, we propose a novel approach for HMS, named Temporal Rate Reduction Clustering ($\text{TR}^2\text{C}$), which jointly learns structured representations and affinity to segment the sequences of frames in video. Specifically, the structured representations learned by $\text{TR}^2\text{C}$ enjoy temporally consistency and are aligned well with a UoS structure, which is favorable for addressing the HMS task. We conduct extensive experiments on five benchmark HMS datasets and achieve state-of-the-art performances with different feature extractors. The code is available at: https://github.com/mengxianghan123/TR2C.