TabKANet: Tabular Data Modeling with Kolmogorov-Arnold Network and Transformer

TL;DR

TabKANet introduces a novel Transformer-based model with a Kolmogorov-Arnold Network for improved numerical and categorical feature encoding, achieving superior or comparable results to existing models on various tabular data tasks.

Contribution

The paper presents the first integration of a Kolmogorov-Arnold Network with Transformers for tabular data modeling, enhancing numerical feature learning.

Findings

Outperforms neural networks on multiple datasets

Achieves performance comparable to GBDT models

Demonstrates stable and superior results across tasks

Abstract

Tabular data is the most common type of data in real-life scenarios. In this study, we propose the TabKANet model for tabular data modeling, which targets the bottlenecks in learning from numerical content. We constructed a Kolmogorov-Arnold Network (KAN) based Numerical Embedding Module and unified numerical and categorical features encoding within a Transformer architecture. TabKANet has demonstrated stable and significantly superior performance compared to Neural Networks (NNs) across multiple public datasets in binary classification, multi-class classification, and regression tasks. Its performance is comparable to or surpasses that of Gradient Boosted Decision Tree models (GBDTs). Our code is publicly available on GitHub: https://github.com/AI-thpremed/TabKANet.