MINR: Implicit Neural Representations with Masked Image Modelling

TL;DR

MINR introduces a novel framework combining implicit neural representations with masked image modeling, resulting in more robust, generalizable, and efficient image reconstruction for self-supervised learning, outperforming existing methods especially on out-of-distribution data.

Contribution

The paper proposes MINR, a new approach that integrates implicit neural representations with masked image modeling to enhance robustness and generalization in self-supervised image learning.

Findings

MINR outperforms MAE in in-domain and out-of-distribution tasks.

MINR reduces model complexity compared to existing methods.

MINR demonstrates versatility across various self-supervised applications.

Abstract

Self-supervised learning methods like masked autoencoders (MAE) have shown significant promise in learning robust feature representations, particularly in image reconstruction-based pretraining task. However, their performance is often strongly dependent on the masking strategies used during training and can degrade when applied to out-of-distribution data. To address these limitations, we introduce the masked implicit neural representations (MINR) framework that synergizes implicit neural representations with masked image modeling. MINR learns a continuous function to represent images, enabling more robust and generalizable reconstructions irrespective of masking strategies. Our experiments demonstrate that MINR not only outperforms MAE in in-domain scenarios but also in out-of-distribution settings, while reducing model complexity. The versatility of MINR extends to various self-supervised learning applications, confirming its utility as a robust and efficient alternative to existing frameworks.