Self-Supervised Continuous Colormap Recovery from a 2D Scalar Field Visualization without a Legend

TL;DR

This paper introduces a self-supervised method to recover continuous colormaps from 2D scalar field visualizations without legends, using a decoupling and reconstruction strategy guided by a differentiable color mapping and smoothness constraints.

Contribution

The authors propose a novel self-supervised approach that extracts continuous colormaps from visualizations without legends, employing a decoupling module, a differentiable reconstruction, and a cubic B-spline based smoothness constraint.

Findings

Effective on synthetic and real-world datasets.

Enables colormap adjustment and transfer.

Generalizes to visualizations with legends and discrete palettes.

Abstract

Recovering a continuous colormap from a single 2D scalar field visualization can be quite challenging, especially in the absence of a corresponding color legend. In this paper, we propose a novel colormap recovery approach that extracts the colormap from a color-encoded 2D scalar field visualization by simultaneously predicting the colormap and underlying data using a decoupling-and-reconstruction strategy. Our approach first separates the input visualization into colormap and data using a decoupling module, then reconstructs the visualization with a differentiable color-mapping module. To guide this process, we design a reconstruction loss between the input and reconstructed visualizations, which serves both as a constraint to ensure strong correlation between colormap and data during training, and as a self-supervised optimizer for fine-tuning the predicted colormap of unseen visualizations during inferencing. To ensure smoothness and correct color ordering in the extracted colormap, we introduce a compact colormap representation using cubic B-spline curves and an associated color order loss. We evaluate our method quantitatively and qualitatively on a synthetic dataset and a collection of real-world visualizations from the VIS30K dataset. Additionally, we demonstrate its utility in two prototype applications -- colormap adjustment and colormap transfer -- and explore its generalization to visualizations with color legends and ones encoded using discrete color palettes.