CodeBoost: Boosting Code LLMs by Squeezing Knowledge from Code Snippets with RL

TL;DR

CodeBoost is a novel post-training framework that enhances code large language models by leveraging abundant code snippets and multiple learning strategies, eliminating the need for labor-intensive human instructions.

Contribution

It introduces a scalable, instruction-free post-training method for code LLMs using diverse code snippets and innovative learning components.

Findings

Consistently improves code LLM performance across benchmarks.

Effective without relying on human-annotated instructions.

Enhances model understanding through diverse training strategies.

Abstract

Code large language models (LLMs) have become indispensable tools for building efficient and automated coding pipelines. Existing models are typically post-trained using reinforcement learning (RL) from general-purpose LLMs using "human instruction-final answer" pairs, where the instructions are usually from manual annotations. However, collecting high-quality coding instructions is both labor-intensive and difficult to scale. On the other hand, code snippets are abundantly available from various sources. This imbalance presents a major bottleneck in instruction-based post-training. We propose CodeBoost, a post-training framework that enhances code LLMs purely from code snippets, without relying on human-annotated instructions. CodeBoost introduces the following key components: (1) maximum-clique curation, which selects a representative and diverse training corpus from code; (2) bi-directional prediction, which enables the model to learn from both forward and backward prediction objectives; (3) error-aware prediction, which incorporates learning signals from both correct and incorrect outputs; (4) heterogeneous augmentation, which diversifies the training distribution to enrich code semantics; and (5) heterogeneous rewarding, which guides model learning through multiple reward types including format correctness and execution feedback from both successes and failures. Extensive experiments across several code LLMs and benchmarks verify that CodeBoost consistently improves performance, demonstrating its effectiveness as a scalable and effective training pipeline.