LLMBoost: Make Large Language Models Stronger with Boosting

TL;DR

LLMBoost introduces a novel ensemble fine-tuning framework for LLMs that leverages intermediate states, hierarchical error correction, and near-parallel inference to improve accuracy and efficiency.

Contribution

It presents a new boosting-inspired method that explicitly utilizes internal model states and a chain training paradigm for enhanced ensemble learning of LLMs.

Findings

Consistently improves accuracy on reasoning tasks

Reduces inference latency significantly

Theoretically guarantees monotonic improvements

Abstract

Ensemble learning of LLMs has emerged as a promising alternative to enhance performance, but existing approaches typically treat models as black boxes, combining the inputs or final outputs while overlooking the rich internal representations and interactions across models.In this work, we introduce LLMBoost, a novel ensemble fine-tuning framework that breaks this barrier by explicitly leveraging intermediate states of LLMs. Inspired by the boosting paradigm, LLMBoost incorporates three key innovations. First, a cross-model attention mechanism enables successor models to access and fuse hidden states from predecessors, facilitating hierarchical error correction and knowledge transfer. Second, a chain training paradigm progressively fine-tunes connected models with an error-suppression objective, ensuring that each model rectifies the mispredictions of its predecessor with minimal additional computation. Third, a near-parallel inference paradigm design pipelines hidden states across models layer by layer, achieving inference efficiency approaching single-model decoding. We further establish the theoretical foundations of LLMBoost, proving that sequential integration guarantees monotonic improvements under bounded correction assumptions. Extensive experiments on commonsense reasoning and arithmetic reasoning tasks demonstrate that LLMBoost consistently boosts accuracy while reducing inference latency.