MeTHanol: Modularized Thinking Language Models with Intermediate Layer Thinking, Decoding and Bootstrapping Reasoning

TL;DR

MeTHanol introduces a modular approach to enhance language models' reasoning by leveraging intermediate layer decoding, dual-layer fine-tuning, and a two-pass inference mechanism, leading to improved cognitive and reflective capabilities.

Contribution

This work presents a novel modular framework with intermediate layer decoding and dual-layer fine-tuning to improve reasoning and self-reflection in large language models.

Findings

Intermediate layer can decode fluent language tokens.

Two-pass inference enhances reasoning and response quality.

Model demonstrates improved cognitive behaviors and adaptability.

Abstract

Current research efforts are focused on enhancing the thinking and reasoning capability of large language model (LLM) by prompting, data-driven emergence and inference-time computation. In this study, we consider stimulating language model's thinking and cognitive abilities from a modular perspective, which mimics the human brain architecture. We select a specific intermediate attention layer with newly implemented language heads. We conduct dual-layer fine-tuning by annotated (query, thought, answer) samples and show that the intermediate layer can also learn to decode fluent and reasonable language tokens. A two-pass inference mechanism is designed to generate thoughts then formal responses. The entire framework is called modularized thinking language model (MeTHanol) which can enhance LLM's cognitive behaviors as indicated by Theory of Mind (ToM) and Vignette-based experiments. Case studies also show that MeTHanol can plan and self-reflect and generate human-like thoughts and answers, even on unseen and open-domain tasks. MeTHanol can also adapt to a personalized prompt and behave as the specified character. Our study holds promise for significant cognitive gains from a modular perspective. Our code, model and data are available at https://bachozean.github.io/methanol-page