Value-Guided Iterative Refinement and the DIQ-H Benchmark for Evaluating VLM Robustness

TL;DR

This paper introduces the DIQ-H benchmark for evaluating vision-language models under adversarial, real-world visual stressors over time, and proposes the VIR framework to improve value alignment and robustness.

Contribution

It presents the first benchmark for continuous, adversarial visual conditions in VLMs and a scalable, value-guided refinement framework to enhance safety and ethical alignment.

Findings

VIR improves value alignment accuracy from 72.2% to 83.3%.

DIQ-H reveals vulnerabilities in error recovery and ethical consistency.

Benchmark simulates real-world visual stressors like motion blur and noise.

Abstract

Vision-Language Models (VLMs) are essential for embodied AI and safety-critical applications, such as robotics and autonomous systems. However, existing benchmarks primarily focus on static or curated visual inputs, neglecting the challenges posed by adversarial conditions, value misalignment, and error propagation in continuous deployment. Current benchmarks either overlook the impact of real-world perturbations, or fail to account for the cumulative effect of inconsistent reasoning over time. To address these gaps, we introduce the Degraded Image Quality Leading to Hallucinations (DIQ-H) benchmark, the first to evaluate VLMs under adversarial visual conditions in continuous sequences. DIQ-H simulates real-world stressors including motion blur, sensor noise, and compression artifacts, and measures how these corruptions lead to persistent errors and misaligned outputs across time. The benchmark explicitly models error propagation and its long-term value consistency. To enhance scalability and reduce costs for safety-critical evaluation, we propose the Value-Guided Iterative Refinement (VIR) framework, which automates the generation of high-quality, ethically aligned ground truth annotations. VGIR leverages lightweight VLMs to detect and refine value misalignment, improving accuracy from 72.2% to 83.3%, representing a 15.3% relative improvement. The DIQ-H benchmark and VGIR framework provide a robust platform for embodied AI safety assessment, revealing vulnerabilities in error recovery, ethical consistency, and temporal value alignment.