DynaTab: Dynamic Feature Ordering as Neural Rewiring for High-Dimensional Tabular Data

TL;DR

DynaTab introduces a neural rewiring approach that dynamically reorders features in high-dimensional tabular data, improving deep learning performance by adapting to dataset complexity.

Contribution

It proposes a novel dynamic feature ordering architecture with a neural rewiring algorithm, enhancing deep learning on high-dimensional tabular data.

Findings

DynaTab outperforms 45 state-of-the-art baselines across 36 datasets.

It achieves statistically significant improvements, especially on high-dimensional data.

The method is compatible with any sequence-sensitive backbone.

Abstract

High-dimensional tabular data lacks a natural feature order, limiting the applicability of permutation-sensitive deep learning models. We propose DynaTab, a dynamic feature ordering-enabled architecture inspired by neural rewiring. We introduce a lightweight criterion that predicts when feature permutation will benefit a dataset by quantifying its intrinsic complexity. DynaTab dynamically reorders features via a neural rewiring algorithm and processes them through a compact, dynamic order-aware combination of separate learned positional embedding, importance-based gating, and masked attention layers, compatible with any sequence-sensitive backbone. Trained end-to-end with bespoke dynamic feature ordering (DFO) and dispersion losses, DynaTab achieves statistically significant gains, particularly on high-dimensional datasets, where it is benchmarked against 45 state-of-the-art baselines across 36 different real-world tabular datasets. Our results position DynaTab as a compelling new paradigm for high-dimensional tabular deep learning.