KMLP: A Scalable Hybrid Architecture for Web-Scale Tabular Data Modeling

TL;DR

KMLP is a scalable hybrid deep learning architecture designed for web-scale tabular data, combining a feature-specific non-linear transformation front-end with a high-order interaction backbone, achieving state-of-the-art results.

Contribution

The paper introduces KMLP, a novel hybrid architecture that effectively models complex large-scale tabular data, outperforming traditional methods and demonstrating scalability.

Findings

KMLP outperforms GBDTs on large-scale benchmarks.

Advantages of KMLP increase with data size.

Validated on industrial datasets with billions of samples.

Abstract

Predictive modeling on web-scale tabular data with billions of instances and hundreds of heterogeneous numerical features faces significant scalability challenges. These features exhibit anisotropy, heavy-tailed distributions, and non-stationarity, creating bottlenecks for models like Gradient Boosting Decision Trees and requiring laborious manual feature engineering. We introduce KMLP, a hybrid deep architecture integrating a shallow Kolmogorov-Arnold Network (KAN) front-end with a Gated Multilayer Perceptron (gMLP) backbone. The KAN front-end uses learnable activation functions to automatically model complex non-linear transformations for each feature, while the gMLP backbone captures high-order interactions. Experiments on public benchmarks and an industrial dataset with billions of samples show KMLP achieves state-of-the-art performance, with advantages over baselines like GBDTs increasing at larger scales, validating KMLP as a scalable deep learning paradigm for large-scale web tabular data.