Learning Transformation-Isomorphic Latent Space for Accurate Hand Pose Estimation

TL;DR

This paper introduces TI-Net, a novel network backbone that constructs a transformation-isomorphic latent space to improve hand pose estimation accuracy by capturing low-level, task-relevant features.

Contribution

The paper proposes TI-Net, a versatile backbone that models geometric transformations in latent space, aligning them with image space to enhance pose estimation accuracy.

Findings

TI-Net achieved a 10% improvement in PA-MPJPE on DexYCB dataset.

TI-Net effectively captures low-level, task-relevant features for pose estimation.

The approach outperforms existing state-of-the-art methods.

Abstract

Vision-based regression tasks, such as hand pose estimation, have achieved higher accuracy and faster convergence through representation learning. However, existing representation learning methods often encounter the following issues: the high semantic level of features extracted from images is inadequate for regressing low-level information, and the extracted features include task-irrelevant information, reducing their compactness and interfering with regression tasks. To address these challenges, we propose TI-Net, a highly versatile visual Network backbone designed to construct a Transformation Isomorphic latent space. Specifically, we employ linear transformations to model geometric transformations in the latent space and ensure that {\rm TI-Net} aligns them with those in the image space. This ensures that the latent features capture compact, low-level information beneficial for pose estimation tasks. We evaluated TI-Net on the hand pose estimation task to demonstrate the network's superiority. On the DexYCB dataset, TI-Net achieved a 10% improvement in the PA-MPJPE metric compared to specialized state-of-the-art (SOTA) hand pose estimation methods. Our code will be released in the future.