Deconfounded Causality-aware Parameter-Efficient Fine-Tuning for Problem-Solving Improvement of LLMs

TL;DR

This paper introduces a causal framework and a parameter-efficient fine-tuning method called Deconfounded Causal Adaptation (DCA) to improve the reasoning abilities of large language models, demonstrating superior performance with minimal additional parameters.

Contribution

The paper proposes a novel causal framework for understanding LLM reasoning and introduces DCA, a PEFT method that enhances reasoning skills with only 1.2 million parameters.

Findings

DCA outperforms baseline methods on multiple benchmarks.

Achieves comparable or better results with significantly fewer parameters.

Enhances LLM reasoning capabilities and reliability.

Abstract

Large Language Models (LLMs) have demonstrated remarkable efficiency in tackling various tasks based on human instructions, but studies reveal that they often struggle with tasks requiring reasoning, such as math or physics. This limitation raises questions about whether LLMs truly comprehend embedded knowledge or merely learn to replicate the token distribution without a true understanding of the content. In this paper, we delve into this problem and aim to enhance the reasoning capabilities of LLMs. First, we investigate if the model has genuine reasoning capabilities by visualizing the text generation process at the attention and representation level. Then, we formulate the reasoning process of LLMs into a causal framework, which provides a formal explanation of the problems observed in the visualization. Finally, building upon this causal framework, we propose Deconfounded Causal Adaptation (DCA), a novel parameter-efficient fine-tuning (PEFT) method to enhance the model's reasoning capabilities by encouraging the model to extract the general problem-solving skills and apply these skills to different questions. Experiments show that our method outperforms the baseline consistently across multiple benchmarks, and with only 1.2M tunable parameters, we achieve better or comparable results to other fine-tuning methods. This demonstrates the effectiveness and efficiency of our method in improving the overall accuracy and reliability of LLMs.