NeuralAnnot: Neural Annotator for 3D Human Mesh Training Sets

TL;DR

NeuralAnnot introduces a neural network-based annotation method for generating high-quality 3D human mesh training data, improving upon traditional optimization-based annotators by leveraging weak supervision and producing more accurate pseudo-GTs.

Contribution

The paper presents NeuralAnnot, a neural network-based annotator that produces better 3D pseudo-GTs for training human mesh regressors, addressing limitations of existing optimization-based methods.

Findings

NeuralAnnot generates more accurate 3D pseudo-GTs than traditional methods.

Training regressors with NeuralAnnot's pseudo-GTs improves 3D human mesh reconstruction.

Publicly available 3D pseudo-GTs facilitate further research.

Abstract

Most 3D human mesh regressors are fully supervised with 3D pseudo-GT human model parameters and weakly supervised with GT 2D/3D joint coordinates as the 3D pseudo-GTs bring great performance gain. The 3D pseudo-GTs are obtained by annotators, systems that iteratively fit 3D human model parameters to GT 2D/3D joint coordinates of training sets in the pre-processing stage of the regressors. The fitted 3D parameters at the last fitting iteration become the 3D pseudo-GTs, used to fully supervise the regressors. Optimization-based annotators, such as SMPLify-X, have been widely used to obtain the 3D pseudo-GTs. However, they often produce wrong 3D pseudo-GTs as they fit the 3D parameters to GT of each sample independently. To overcome the limitation, we present NeuralAnnot, a neural network-based annotator. The main idea of NeuralAnnot is to employ a neural network-based regressor and dedicate it for the annotation. Assuming no 3D pseudo-GTs are available, NeuralAnnot is weakly supervised with GT 2D/3D joint coordinates of training sets. The testing results on the same training sets become 3D pseudo-GTs, used to fully supervise the regressors. We show that 3D pseudo-GTs of NeuralAnnot are highly beneficial to train the regressors. We made our 3D pseudo-GTs publicly available.