CaptchaMind: Training CAPTCHA Solvers via Reinforcement Learning with Explicit Reasoning Supervision

TL;DR

CaptchaMind is a reinforcement learning-based CAPTCHA solver trained with explicit reasoning supervision, achieving high success rates on a new large-scale benchmark and outperforming existing methods.

Contribution

The paper introduces CaptchaBench, a large-scale CAPTCHA dataset with detailed annotations, and proposes CaptchaMind, a novel RL-based solver with explicit reasoning supervision.

Findings

CaptchaMind achieves 82.9% success rate on benchmark tasks.

Existing methods fail on tasks requiring fine-grained visual reasoning.

CaptchaMind outperforms all existing methods on real-world CAPTCHA instances.

Abstract

CAPTCHAs are widely deployed as human verification mechanisms and frequently block intelligent agents from completing end-to-end automation in real-world web environments. Solving modern CAPTCHAs requires robust multi-step visual reasoning and interaction capabilities, yet training-based approaches have remained absent due to the lack of large-scale training data and process-level annotations. We introduce CaptchaBench, the first CAPTCHA benchmark designed to support large-scale training, comprising 16,000 programmatically generated samples across eight task categories with detailed region and process-level annotations. Systematic evaluation on CaptchaBench reveals that existing methods fail consistently on tasks requiring fine-grained visual detail capture and region-level comparison. We therefore present CaptchaMind, an RL-based solver trained with explicit reasoning process supervision, achieving 82.9% average success rate across eight tasks and 71.0% on real-world instances, substantially outperforming all existing methods without closed-source APIs.