CoCoP: Enhancing Text Classification with LLM through Code Completion Prompt

TL;DR

This paper introduces CoCoP, a novel prompt technique that transforms text classification into a code completion task, significantly improving performance of LLMs across datasets by leveraging their code-related capabilities.

Contribution

The paper proposes CoCoP, a new method that enhances text classification by converting it into a code completion problem, utilizing LLMs' code skills for better accuracy.

Findings

CoCoP improves SST2 accuracy by over 20%.

Using code models like CodeLLaMA with CoCoP achieves comparable results to few-shot learning.

CoCoP requires only one-tenth of the model size for similar performance.

Abstract

Text classification is a fundamental task in natural language processing (NLP), and large language models (LLMs) have demonstrated their capability to perform this task across various domains. However, the performance of LLMs heavily depends on the quality of their input prompts. Recent studies have also shown that LLMs exhibit remarkable results in code-related tasks. To leverage the capabilities of LLMs in text classification, we propose the Code Completion Prompt (CoCoP) method, which transforms the text classification problem into a code completion task. CoCoP significantly improves text classification performance across diverse datasets by utilizing LLMs' code-completion capability. For instance, CoCoP enhances the accuracy of the SST2 dataset by more than 20%. Moreover, when CoCoP integrated with LLMs specifically designed for code-related tasks (code models), such as CodeLLaMA, this method demonstrates better or comparable performance to few-shot learning techniques while using only one-tenth of the model size. The source code of our proposed method will be available to the public upon the acceptance of the paper.