2D Neural Fields with Learned Discontinuities

TL;DR

This paper introduces a novel neural field model that jointly captures images and their discontinuities, significantly improving denoising, super-resolution, and discontinuity detection over existing methods.

Contribution

The paper proposes a discontinuous neural field model that treats all mesh edges as potential discontinuities, enabling joint image approximation and discontinuity recovery without predefined meshes.

Findings

Achieves over 5dB improvement in denoising

Achieves over 10dB improvement in super-resolution

Chamfer distances 3.5x closer to ground truth

Abstract

Effective representation of 2D images is fundamental in digital image processing, where traditional methods like raster and vector graphics struggle with sharpness and textural complexity respectively. Current neural fields offer high-fidelity and resolution independence but require predefined meshes with known discontinuities, restricting their utility. We observe that by treating all mesh edges as potential discontinuities, we can represent the magnitude of discontinuities with continuous variables and optimize. Based on this observation, we introduce a novel discontinuous neural field model that jointly approximate the target image and recovers discontinuities. Through systematic evaluations, our neural field demonstrates superior performance in denoising and super-resolution tasks compared to InstantNGP, achieving improvements of over 5dB and 10dB, respectively. Our model also outperforms Mumford-Shah-based methods in accurately capturing discontinuities, with Chamfer distances 3.5x closer to the ground truth. Additionally, our approach shows remarkable capability in handling complex artistic drawings and natural images.