Learning to Solve and Verify: A Self-Play Framework for Code and Test Generation

TL;DR

This paper introduces Sol-Ver, a self-play framework where a language model iteratively verifies and improves its own code and test generation, leading to significant performance gains without human annotations.

Contribution

The paper presents a novel self-play approach for code and test generation that enhances model capabilities through iterative self-verification, avoiding reliance on larger models or human data.

Findings

Achieved 19.63% improvement in code generation accuracy.

Achieved 17.49% improvement in test generation accuracy.

Demonstrated effectiveness on MBPP and LiveCodeBench benchmarks.

Abstract

Recent advances in large language models (LLMs) have improved their performance on coding benchmarks. However, improvement is plateauing due to the exhaustion of readily available high-quality data. Prior work has shown the potential of synthetic self-instruct data, but naively training on a model's own outputs can cause error accumulation, especially in coding tasks, where generalization may collapse due to overly simple or erroneous training data, highlighting the need for rigorous quality checks on synthetic data. In this work, we explore an effective approach whereby the model itself verifies the correctness of its own data. We thus propose Sol-Ver, a self-play solver-verifier framework that jointly improves a single model's code and test generation capacity. By iteratively refining code (LLM-as-a-solver) and tests (LLM-as-a-verifier) together, we boost both capabilities without relying on human annotations or larger teacher models. Experiments with the Llama 3.1 8B model demonstrate substantial performance enhancements, achieving average relative improvements of 19.63% in code generation and 17.49% in test generation on MBPP and LiveCodeBench.