Collaborating Domain-shared and Target-specific Feature Clustering for Cross-domain 3D Action Recognition

TL;DR

This paper introduces CoDT, a novel method for cross-domain 3D action recognition that effectively clusters target data by leveraging both domain-shared and target-specific features through collaborative clustering and contrastive learning.

Contribution

The paper proposes a new collaborative clustering approach that combines domain-shared and target-specific features for open-set cross-domain 3D action recognition.

Findings

Effective clustering of target data demonstrated on multiple datasets.

Significant improvement over existing methods in cross-domain accuracy.

Robust pseudo label generation via online clustering.

Abstract

In this work, we consider the problem of cross-domain 3D action recognition in the open-set setting, which has been rarely explored before. Specifically, there is a source domain and a target domain that contain the skeleton sequences with different styles and categories, and our purpose is to cluster the target data by utilizing the labeled source data and unlabeled target data. For such a challenging task, this paper presents a novel approach dubbed CoDT to collaboratively cluster the domain-shared features and target-specific features. CoDT consists of two parallel branches. One branch aims to learn domain-shared features with supervised learning in the source domain, while the other is to learn target-specific features using contrastive learning in the target domain. To cluster the features, we propose an online clustering algorithm that enables simultaneous promotion of robust pseudo label generation and feature clustering. Furthermore, to leverage the complementarity of domain-shared features and target-specific features, we propose a novel collaborative clustering strategy to enforce pair-wise relationship consistency between the two branches. We conduct extensive experiments on multiple cross-domain 3D action recognition datasets, and the results demonstrate the effectiveness of our method.