ReFit: Recurrent Fitting Network for 3D Human Recovery

TL;DR

ReFit is a neural network architecture that iteratively refines 3D human reconstructions from a single image using a feedback loop, achieving faster training and improved accuracy over previous methods.

Contribution

It introduces a feedback-update loop in a neural network for 3D human reconstruction, combining optimization principles with deep learning for enhanced performance.

Findings

Faster training compared to previous regression models.

State-of-the-art accuracy on standard benchmarks.

Applicable to multi-view and single-view shape fitting.

Abstract

We present Recurrent Fitting (ReFit), a neural network architecture for single-image, parametric 3D human reconstruction. ReFit learns a feedback-update loop that mirrors the strategy of solving an inverse problem through optimization. At each iterative step, it reprojects keypoints from the human model to feature maps to query feedback, and uses a recurrent-based updater to adjust the model to fit the image better. Because ReFit encodes strong knowledge of the inverse problem, it is faster to train than previous regression models. At the same time, ReFit improves state-of-the-art performance on standard benchmarks. Moreover, ReFit applies to other optimization settings, such as multi-view fitting and single-view shape fitting. Project website: https://yufu-wang.github.io/refit_humans/