Test-time local training of neural network for tabular data

TL;DR

This paper introduces a method to improve neural network predictions for tabular data by adjusting the model during inference based on local data structures.

Contribution

The novel contribution is a test-time local training method tailored for tabular data to enhance generalization.

Findings

The proposed method improves generalization performance on tabular datasets.

Local fine-tuning with nearest neighbors enhances prediction accuracy during inference.

Abstract

Generally, a neural network is globally trained using the dataset provided in the training phase to optimize its parameters. The trained neural network is then used to make predictions for query instances in the inference phase. This global learning approach leads to a neural network that performs well universally across various query instances. However, it may overlook local structures in some low-density data regions, potentially degrading generalization performance in these regions. Although several test-time adaptation methods have been explored in recent years, they are typically designed for vision domains and are not intended for or do not readily transfer to tabular data. In this study, we propose a test-time local training method, specifically tailored for tabular data, to make the neural network better reflect the local structure around the query instance during the inference phase. Given a query instance, the proposed method finds the nearest neighbors of that instance from the training dataset. It then localizes the globally trained neural network by fine-tuning with these nearest neighbors to better accommodate the local structure around the query instance. The localized neural network is finally used to make a prediction for the query instance. Through experiments conducted on tabular benchmark datasets for regression and classification tasks, we demonstrate that the proposed method significantly enhances the generalization ability of neural networks.

Figures