Self-supervised Analogical Learning using Language Models

TL;DR

This paper introduces SAL, a self-supervised framework that enhances language models' reasoning by teaching them to transfer high-quality solutions across cases, improving performance and generalization on reasoning tasks.

Contribution

The paper presents SAL, a novel analogical learning method that explicitly trains models to transfer solutions, addressing reasoning inconsistency issues in language models.

Findings

Models with SAL outperform base models by 2-20% on reasoning benchmarks.

SAL improves model generalization and controllability.

Models trained with SAL better handle unfamiliar reasoning cases.

Abstract

Large language models have been shown to suffer from reasoning inconsistency issues. That is, they fail more in situations unfamiliar to the training data, even though exact or very similar reasoning paths exist in more common cases that they can successfully solve. Such observations motivate us to propose methods that encourage models to understand the high-level and abstract reasoning processes during training instead of only the final answer. This way, models can transfer the exact solution to similar cases, regardless of their relevance to the pre-training data distribution. In this work, we propose SAL, a self-supervised analogical learning framework. SAL mimics the human analogy process and trains models to explicitly transfer high-quality symbolic solutions from cases that they know how to solve to other rare cases in which they tend to fail more. We show that the resulting models after SAL learning outperform base language models on a wide range of reasoning benchmarks, such as StrategyQA, GSM8K, and HotpotQA, by 2% to 20%. At the same time, we show that our model is more generalizable and controllable through analytical studies.