3D Shape Completion with Test-Time Training

TL;DR

This paper introduces a test-time training approach for 3D shape completion that separately predicts fractured and restored parts, finetuning the model during inference to improve accuracy and reduce artifacts.

Contribution

It proposes a novel method that predicts fractured and restored parts separately with interconnected predictions, utilizing test-time training for improved shape completion.

Findings

Significant reduction in artifacts around fracture boundaries.

Improved chamfer distance metrics across eight shape categories.

Effective finetuning during inference enhances shape restoration accuracy.

Abstract

This work addresses the problem of \textit{shape completion}, i.e., the task of restoring incomplete shapes by predicting their missing parts. While previous works have often predicted the fractured and restored shape in one step, we approach the task by separately predicting the fractured and newly restored parts, but ensuring these predictions are interconnected. We use a decoder network motivated by related work on the prediction of signed distance functions (DeepSDF). In particular, our representation allows us to consider test-time-training, i.e., finetuning network parameters to match the given incomplete shape more accurately during inference. While previous works often have difficulties with artifacts around the fracture boundary, we demonstrate that our overfitting to the fractured parts leads to significant improvements in the restoration of eight different shape categories of the ShapeNet data set in terms of their chamfer distances.