G1: Bootstrapping Perception and Reasoning Abilities of Vision-Language Model via Reinforcement Learning

TL;DR

This paper introduces VLM-Gym, a reinforcement learning environment for vision-language models, demonstrating that perception and reasoning abilities mutually improve through self-evolution and fine-tuning, leading to superior performance in visual games.

Contribution

The paper presents VLM-Gym for scalable multi-game training and introduces G1 models with perception-enhanced priors, achieving state-of-the-art results and revealing mutual bootstrap effects between perception and reasoning.

Findings

G1 models outperform their teachers and proprietary models.

Perception and reasoning abilities mutually bootstrap during RL training.

VLM-Gym enables scalable multi-game reinforcement learning for vision-language models.

Abstract

Vision-Language Models (VLMs) excel in many direct multimodal tasks but struggle to translate this prowess into effective decision-making within interactive, visually rich environments like games. This ``knowing-doing'' gap significantly limits their potential as autonomous agents, as leading VLMs often performing badly in simple games. To address this, we introduce VLM-Gym, a curated reinforcement learning (RL) environment featuring diverse visual games with unified interfaces and adjustable, compositional difficulty, specifically designed for scalable multi-game parallel training. Leveraging VLM-Gym, we train G0 models using pure RL-driven self-evolution, which demonstrate emergent perception and reasoning patterns. To further mitigate challenges arising from game diversity, we develop G1 models. G1 incorporates a perception-enhanced cold start prior to RL fine-tuning. Our resulting G1 models consistently surpass their teacher across all games and outperform leading proprietary models like Claude-3.7-Sonnet-Thinking. Systematic analysis reveals an intriguing finding: perception and reasoning abilities mutually bootstrap each other throughout the RL training process. Source code including VLM-Gym and RL training are released at https://github.com/chenllliang/G1 to foster future research in advancing VLMs as capable interactive agents.