FourierMask: Instance Segmentation using Fourier Mapping in Implicit Neural Networks

TL;DR

FourierMask introduces a novel approach combining Fourier series with implicit neural representations to generate high-resolution instance segmentation masks, achieving competitive results on MS COCO.

Contribution

It proposes a new Fourier-based implicit neural network method for instance segmentation that adapts Fourier coefficients per object and improves mask resolution.

Findings

Achieves competitive results on MS COCO at standard resolution.

Surpasses Mask R-CNN on higher resolution masks.

Uses a Fourier-based implicit representation with a secondary renderer for quality enhancement.

Abstract

We present FourierMask, which employs Fourier series combined with implicit neural representations to generate instance segmentation masks. We apply a Fourier mapping (FM) to the coordinate locations and utilize the mapped features as inputs to an implicit representation (coordinate-based multi-layer perceptron (MLP)). FourierMask learns to predict the coefficients of the FM for a particular instance, and therefore adapts the FM to a specific object. This allows FourierMask to be generalized to predict instance segmentation masks from natural images. Since implicit functions are continuous in the domain of input coordinates, we illustrate that by sub-sampling the input pixel coordinates, we can generate higher resolution masks during inference. Furthermore, we train a renderer MLP (FourierRend) on the uncertain predictions of FourierMask and illustrate that it significantly improves the quality of the masks. FourierMask shows competitive results on the MS COCO dataset compared to the baseline Mask R-CNN at the same output resolution and surpasses it on higher resolution.