Chain-of-Conceptual-Thought Elicits Daily Conversation in Large Language Models

TL;DR

This paper introduces Chain of Conceptual Thoughts (CoCT), a new prompting paradigm for large language models that improves open-domain conversation by structuring reasoning around concepts, emotions, and strategies.

Contribution

It proposes a hierarchical prompting method called CoCT that enhances LLM performance in open-domain and emotional conversations, outperforming existing prompt-based techniques.

Findings

CoCT outperforms baseline prompting methods in evaluations.

Hierarchical concept-based prompting improves reasoning in open-domain tasks.

Effective in both in-domain and out-of-domain conversational settings.

Abstract

Chain-of-Thought (CoT) is widely applied to enhance the LLM capability in math, coding and reasoning tasks. However, its performance is limited for open-domain tasks, when there are no clearly defined reasoning steps or logical transitions. To mitigate such challenges, we propose a new prompt-based paradigm called Chain of Conceptual Thoughts (CoCT), which suggests the LLM first to produce the tag of concepts, then complete the detailed content following the concept. To encourage this hierarchical way of thinking, we implement the concepts with emotions, strategies and topics. We experiment with this paradigm in daily and emotional support conversations, covering tasks with both in-domain and out-of-domain concept settings. Automatic, human, and LLM-based evaluations reveal that CoCT surpasses several prompt-based baselines such as self-refine, ECoT, SoT and RAG, suggesting a potential solution of LLM prompting paradigm for a wider scope of tasks.