LLM Empowered Prototype Learning for Zero and Few-Shot Tasks on Tabular Data

TL;DR

This paper introduces a novel LLM-based prototype learning framework for zero and few-shot tabular data tasks, leveraging task and feature descriptions to generate prototypes without training classifiers or fine-tuning models.

Contribution

It proposes an example-free prompt method for LLMs to generate feature-based prototypes, enabling scalable zero and few-shot learning on tabular data without classifier training.

Findings

Effective in zero-shot tabular classification tasks

Outperforms baseline methods in few-shot scenarios

Scalable and robust framework for tabular data learning

Abstract

Recent breakthroughs in large language models (LLMs) have opened the door to in-depth investigation of their potential in tabular data modeling. However, effectively utilizing advanced LLMs in few-shot and even zero-shot scenarios is still challenging. To this end, we propose a novel LLM-based prototype estimation framework for tabular learning. Our key idea is to query the LLM to generate feature values based example-free prompt, which solely relies on task and feature descriptions. With the feature values generated by LLM, we can build a zero-shot prototype in a training-free manner, which can be further enhanced by fusing few-shot samples, avoiding training a classifier or finetuning the LLMs. Thanks to the example-free prompt and prototype estimation, ours bypasses the constraints brought by the example-based prompt, providing a scalable and robust framework. Extensive experiments demonstrate the effectiveness of ours in zero and few-shot tabular learning.