Complementary Learning Approach for Text Classification using Large Language Models

TL;DR

This paper introduces a cost-efficient, structured approach that leverages large language models in text classification tasks, enhancing human-machine collaboration and interpretability in analyzing pharmaceutical press releases.

Contribution

It presents a novel methodology combining chain of thought and few-shot prompting to improve human-machine team performance in quantitative research.

Findings

Effective interrogation of rating discrepancies in 1,934 press releases

Demonstrates cost-efficient integration of LLMs in qualitative and quantitative analysis

Highlights strategies to manage LLM weaknesses using low-cost techniques

Abstract

In this study, we propose a structured methodology that utilizes large language models (LLMs) in a cost-efficient and parsimonious manner, integrating the strengths of scholars and machines while offsetting their respective weaknesses. Our methodology, facilitated through a chain of thought and few-shot learning prompting from computer science, extends best practices for co-author teams in qualitative research to human-machine teams in quantitative research. This allows humans to utilize abductive reasoning and natural language to interrogate not just what the machine has done but also what the human has done. Our method highlights how scholars can manage inherent weaknesses OF LLMs using careful, low-cost techniques. We demonstrate how to use the methodology to interrogate human-machine rating discrepancies for a sample of 1,934 press releases announcing pharmaceutical alliances (1990-2017).