# Summarize-Exemplify-Reflect: Data-driven Insight Distillation Empowers LLMs for Few-shot Tabular Classification

## TL;DR

InsightTab introduces a data-driven insight distillation framework that enhances large language models' performance in few-shot tabular classification by integrating rule summarization, exemplification, and reflection, inspired by human learning.

## Contribution

The paper presents a novel insight distillation framework, InsightTab, which combines rule summarization, exemplification, and reflection to improve LLMs' effectiveness on tabular data tasks.

## Key findings

- Consistent performance improvements over state-of-the-art methods.
- Effective leveraging of labeled data and bias management.
- Validation of the distillation principles through ablation studies.

## Abstract

Recent studies show the promise of large language models (LLMs) for few-shot tabular classification but highlight challenges due to the variability in structured data. To address this, we propose distilling data into actionable insights to enable robust and effective classification by LLMs. Drawing inspiration from human learning processes, we introduce InsightTab, an insight distillation framework guided by principles of divide-and-conquer, easy-first, and reflective learning. Our approach integrates rule summarization, strategic exemplification, and insight reflection through deep collaboration between LLMs and data modeling techniques. The obtained insights enable LLMs to better align their general knowledge and capabilities with the particular requirements of specific tabular tasks. We extensively evaluate InsightTab on nine datasets. The results demonstrate consistent improvement over state-of-the-art methods. Ablation studies further validate the principle-guided distillation process, while analyses emphasize InsightTab's effectiveness in leveraging labeled data and managing bias.