ZAYAN: Disentangled Contrastive Transformer for Tabular Remote Sensing Data

TL;DR

ZAYAN introduces a self-supervised, feature-centric contrastive learning framework using a Transformer for tabular remote sensing data, improving accuracy and robustness without relying on labels.

Contribution

It proposes a novel feature-level contrastive learning approach with dynamic feature encoding, enhancing representation quality for tabular remote sensing data.

Findings

Outperforms baseline models on eight remote sensing datasets.

Achieves higher accuracy and robustness under label scarcity.

Demonstrates effective generalization across diverse remote sensing tasks.

Abstract

Learning informative representations from tabular data in remote sensing and environmental science is challenging due to heterogeneity, scarce labels, and redundancy among features. We present ZAYAN (Zero-Anchor dYnamic feAture eNcoding), a self-supervised, feature-centric contrastive framework for tabular data. ZAYAN performs contrastive learning at the feature rather than sample level, removing the need for explicit anchor selection and any reliance on class labels, while encouraging a redundancy-minimized, disentangled embedding space. The framework has two modules: ZAYAN-CL, which pretrains feature embeddings via a zero-anchor contrastive objective with dynamic perturbations and masking, and ZAYAN-T, a Transformer that conditions on these embeddings for downstream classification. Across eight datasets, including six remote-sensing tabular benchmarks and two remote-sensing-driven flood-prediction tables from satellite and GIS products, ZAYAN achieves superior accuracy, robustness, and generalization over tabular deep learning baselines, with consistent gains under label scarcity and distribution shift. These results indicate that feature-level contrastive learning and dynamic feature encoding provide an effective recipe for learning from tabular sensing data.