Detecting Multimodal Situations with Insufficient Context and Abstaining from Baseless Predictions

TL;DR

This paper introduces a method to detect and abstain from answering in vision-language tasks when the context is insufficient, improving model trustworthiness and reducing hallucinations.

Contribution

It proposes a novel context-aware abstention detector (CARA) and a context selection module to enhance evidence-based predictions in vision-language understanding benchmarks.

Findings

CARA generalizes well to unseen benchmarks

Significant accuracy improvements with context-aware abstention

Curated CASE set for benchmarking insufficient context detection

Abstract

Despite the widespread adoption of Vision-Language Understanding (VLU) benchmarks such as VQA v2, OKVQA, A-OKVQA, GQA, VCR, SWAG, and VisualCOMET, our analysis reveals a pervasive issue affecting their integrity: these benchmarks contain samples where answers rely on assumptions unsupported by the provided context. Training models on such data foster biased learning and hallucinations as models tend to make similar unwarranted assumptions. To address this issue, we collect contextual data for each sample whenever available and train a context selection module to facilitate evidence-based model predictions. Strong improvements across multiple benchmarks demonstrate the effectiveness of our approach. Further, we develop a general-purpose Context-AwaRe Abstention (CARA) detector to identify samples lacking sufficient context and enhance model accuracy by abstaining from responding if the required context is absent. CARA exhibits generalization to new benchmarks it wasn't trained on, underscoring its utility for future VLU benchmarks in detecting or cleaning samples with inadequate context. Finally, we curate a Context Ambiguity and Sufficiency Evaluation (CASE) set to benchmark the performance of insufficient context detectors. Overall, our work represents a significant advancement in ensuring that vision-language models generate trustworthy and evidence-based outputs in complex real-world scenarios.