HACMatch Semi-Supervised Rotation Regression with Hardness-Aware Curriculum Pseudo Labeling

TL;DR

This paper introduces a semi-supervised rotation regression method that uses a hardness-aware curriculum and structured data augmentation to improve 3D rotation estimation from 2D images, especially with limited labeled data.

Contribution

It proposes a novel hardness-aware curriculum learning framework with multi-stage and adaptive strategies for pseudo-label selection in semi-supervised rotation regression.

Findings

Outperforms existing methods on PASCAL3D+ and ObjectNet3D datasets.

Effective in low-data regimes for rotation estimation.

Structured augmentation enhances feature diversity while maintaining geometric integrity.

Abstract

Regressing 3D rotations of objects from 2D images is a crucial yet challenging task, with broad applications in autonomous driving, virtual reality, and robotic control. Existing rotation regression models often rely on large amounts of labeled data for training or require additional information beyond 2D images, such as point clouds or CAD models. Therefore, exploring semi-supervised rotation regression using only a limited number of labeled 2D images is highly valuable. While recent work FisherMatch introduces semi-supervised learning to rotation regression, it suffers from rigid entropy-based pseudo-label filtering that fails to effectively distinguish between reliable and unreliable unlabeled samples. To address this limitation, we propose a hardness-aware curriculum learning framework that dynamically selects pseudo-labeled samples based on their difficulty, progressing from easy to complex examples. We introduce both multi-stage and adaptive curriculum strategies to replace fixed-threshold filtering with more flexible, hardness-aware mechanisms. Additionally, we present a novel structured data augmentation strategy specifically tailored for rotation estimation, which assembles composite images from augmented patches to introduce feature diversity while preserving critical geometric integrity. Comprehensive experiments on PASCAL3D+ and ObjectNet3D demonstrate that our method outperforms existing supervised and semi-supervised baselines, particularly in low-data regimes, validating the effectiveness of our curriculum learning framework and structured augmentation approach.