Dialectical Behavior Therapy Approach to LLM Prompting

TL;DR

This paper introduces a novel prompting strategy inspired by Dialectical Behavioral Therapy to enhance reasoning in large language models, especially smaller ones, showing significant accuracy improvements across multiple datasets.

Contribution

The paper presents a new DBT-inspired prompting method that improves reasoning accuracy in LLMs, particularly smaller models, outperforming traditional chain-of-thought prompting techniques.

Findings

7% accuracy increase on StrategyQA with 8b model

4.8% accuracy increase on Aqua dataset with 8b model

16.2% accuracy increase on StrategyQA with 14b model

Abstract

Large language models demonstrated state-of-the-art results on various reasoning tasks when applying the chain-of-thought (CoT) prompting technique. CoT prompting guides the model into breaking tasks into a few intermediate steps and provides step-by-step demonstrations. However, solving complex reasoning tasks remains a challenge. In this paper, we propose a novel prompting strategy inspired by Dialectical Behavioral Therapy (DBT). DBT, a form of cognitive-behavioral therapy, aims to help individuals cope with stress by developing a system of reasoning. We applied DBT's basic concepts of shaping dialog to construct prompts and conducted experiments on different datasets and LLMs with various numbers of parameters. Our results show that prompts crafted with DBT techniques significantly improve results on smaller models, achieving a 7% increase in accuracy on the StrategyQA, 4.8% on Aqua dataset using 8b parameters model, and a 16.2% increase on the StrategyQA, 5.3% on GSM8K dataset with 14b parameters model.