Hyper-Transformer for Amodal Completion

TL;DR

The paper introduces H-TAN, a hyper transformer-based framework that directly learns shape priors for amodal object completion, achieving superior results on multiple benchmark datasets.

Contribution

It proposes a novel hyper transformer framework with dynamic convolution for efficient shape prior learning and amodal mask prediction, improving over traditional methods.

Findings

H-TAN outperforms existing methods on KINS, COCOA-cls, and D2SA datasets.

The hyper transformer effectively learns shape priors and enhances mask accuracy.

The model demonstrates robustness and stability across different benchmarks.

Abstract

Amodal object completion is a complex task that involves predicting the invisible parts of an object based on visible segments and background information. Learning shape priors is crucial for effective amodal completion, but traditional methods often rely on two-stage processes or additional information, leading to inefficiencies and potential error accumulation. To address these shortcomings, we introduce a novel framework named the Hyper-Transformer Amodal Network (H-TAN). This framework utilizes a hyper transformer equipped with a dynamic convolution head to directly learn shape priors and accurately predict amodal masks. Specifically, H-TAN uses a dual-branch structure to extract multi-scale features from both images and masks. The multi-scale features from the image branch guide the hyper transformer in learning shape priors and in generating the weights for dynamic convolution tailored to each instance. The dynamic convolution head then uses the features from the mask branch to predict precise amodal masks. We extensively evaluate our model on three benchmark datasets: KINS, COCOA-cls, and D2SA, where H-TAN demonstrated superior performance compared to existing methods. Additional experiments validate the effectiveness and stability of the novel hyper transformer in our framework.