Robust CAPTCHA Using Audio Illusions in the Era of Large Language Models: from Evaluation to Advances

TL;DR

This paper evaluates the vulnerability of existing audio CAPTCHAs to advanced AI models and introduces IllusionAudio, a new approach leveraging audio illusions that is robust against such attacks and maintains high human usability.

Contribution

The paper presents a comprehensive evaluation framework for audio CAPTCHA security and introduces IllusionAudio, a novel CAPTCHA method using audio illusions to enhance robustness against AI-based attacks.

Findings

Most existing audio CAPTCHAs are vulnerable to LALM and ASR attacks.

IllusionAudio defeats all tested AI attacks while maintaining 100% human pass rate.

The evaluation framework ACEval effectively measures CAPTCHA robustness.

Abstract

CAPTCHAs are widely used by websites to block bots and spam by presenting challenges that are easy for humans but difficult for automated programs to solve. To improve accessibility, audio CAPTCHAs are designed to complement visual ones. However, the robustness of audio CAPTCHAs against advanced Large Audio Language Models (LALMs) and Automatic Speech Recognition (ASR) models remains unclear. In this paper, we introduce AI-CAPTCHA, a unified framework that offers (i) an evaluation framework, ACEval, which includes advanced LALM- and ASR-based solvers, and (ii) a novel audio CAPTCHA approach, IllusionAudio, leveraging audio illusions. Through extensive evaluations of seven widely deployed audio CAPTCHAs, we show that most existing methods can be solved with high success rates by advanced LALMs and ASR models, exposing critical security weaknesses. To address these vulnerabilities, we design a new audio CAPTCHA approach, IllusionAudio, which exploits perceptual illusion cues rooted in human auditory mechanisms. Extensive experiments demonstrate that our method defeats all tested LALM- and ASR-based attacks while achieving a 100% human pass rate, significantly outperforming existing audio CAPTCHA methods.