Large Language Model Enhanced Machine Learning Estimators for Classification

TL;DR

This paper explores integrating large language models with classical machine learning classifiers to improve prediction accuracy, demonstrating significant gains across various datasets and transfer learning scenarios.

Contribution

It introduces novel methods for combining LLMs with traditional classifiers, enhancing their performance in classification tasks.

Findings

LLM integration improves classification accuracy

Significant performance gains observed across datasets

Effective in transfer learning scenarios

Abstract

Pre-trained large language models (LLM) have emerged as a powerful tool for simulating various scenarios and generating output given specific instructions and multimodal input. In this work, we analyze the specific use of LLM to enhance a classical supervised machine learning method for classification problems. We propose a few approaches to integrate LLM into a classical machine learning estimator to further enhance the prediction performance. We examine the performance of the proposed approaches through both standard supervised learning binary classification tasks, and a transfer learning task where the test data observe distribution changes compared to the training data. Numerical experiments using four publicly available datasets are conducted and suggest that using LLM to enhance classical machine learning estimators can provide significant improvement on prediction performance.