NeurTV: Total Variation on the Neural Domain

TL;DR

NeurTV introduces a novel total variation regularization on the neural domain, leveraging deep neural networks to better capture local data correlations without discretization errors, applicable to various data types.

Contribution

The paper proposes NeurTV, a new TV regularization defined on the neural domain using DNN derivatives, overcoming limitations of classical TV on the pixel domain.

Findings

NeurTV reduces discretization errors compared to classical TV.

NeurTV effectively handles both meshgrid and non-meshgrid data.

Numerical experiments demonstrate improved data correlation capture across diverse datasets.

Abstract

Recently, we have witnessed the success of total variation (TV) for many imaging applications. However, traditional TV is defined on the original pixel domain, which limits its potential. In this work, we suggest a new TV regularization defined on the neural domain. Concretely, the discrete data is implicitly and continuously represented by a deep neural network (DNN), and we use the derivatives of DNN outputs w.r.t. input coordinates to capture local correlations of data. As compared with classical TV on the original domain, the proposed TV on the neural domain (termed NeurTV) enjoys the following advantages. First, NeurTV is free of discretization error induced by the discrete difference operator. Second, NeurTV is not limited to meshgrid but is suitable for both meshgrid and non-meshgrid data. Third, NeurTV can more exactly capture local correlations across data for any direction and any order of derivatives attributed to the implicit and continuous nature of neural domain. We theoretically reinterpret NeurTV under the variational approximation framework, which allows us to build the connection between NeurTV and classical TV and inspires us to develop variants (e.g., space-variant NeurTV). Extensive numerical experiments with meshgrid data (e.g., color and hyperspectral images) and non-meshgrid data (e.g., point clouds and spatial transcriptomics) showcase the effectiveness of the proposed methods.