Rationales Are Not Silver Bullets: Measuring the Impact of Rationales on Model Performance and Reliability

TL;DR

This paper critically examines the effects of rationale augmentation in language models, revealing that rationales can both harm and enhance performance and reliability depending on task difficulty, challenging previous assumptions.

Contribution

It provides a comprehensive analysis showing that rationales do not always improve models and introduces new insights into their impact on performance and reliability.

Findings

Rationales can sometimes decrease model performance.

Rationales can improve model reliability, sometimes surpassing untrained models.

Performance and reliability improvements are linked to task difficulty.

Abstract

Training language models with rationales augmentation has been shown to be beneficial in many existing works. In this paper, we identify that such a prevailing view does not hold consistently. We conduct comprehensive investigations to thoroughly inspect the impact of rationales on model performance as well as a novel perspective of model reliability. The results lead to several key findings that add new insights upon existing understandings: 1) Rationales can, at times, deteriorate model performance; 2) Rationales can, at times, improve model reliability, even outperforming their untrained counterparts; 3) A linear correspondence exists in between the performance and reliability improvements, while both are driven by the intrinsic difficulty of the task. These findings provide informative regulations on the broad utilization of rationales and raise critical implications on the procedure of explicitly aligning language models with implicit human thoughts. Codes can be found at https://github.com/Ignoramus0817/rationales.