Your AI-Generated Image Detector Can Secretly Achieve SOTA Accuracy, If Calibrated

TL;DR

This paper introduces a simple, theoretically grounded post-hoc calibration method that improves AI-generated image detection accuracy under distributional shifts without retraining the model.

Contribution

We propose a learnable scalar calibration technique based on Bayesian decision theory that enhances detection robustness against distributional shifts without retraining.

Findings

Significant accuracy improvements on challenging benchmarks

Robustness to distributional shifts without retraining

Lightweight calibration method compatible with existing models

Abstract

Despite being trained on balanced datasets, existing AI-generated image detectors often exhibit systematic bias at test time, frequently misclassifying fake images as real. We hypothesize that this behavior stems from distributional shift in fake samples and implicit priors learned during training. Specifically, models tend to overfit to superficial artifacts that do not generalize well across different generation methods, leading to a misaligned decision threshold when faced with test-time distribution shift. To address this, we propose a theoretically grounded post-hoc calibration framework based on Bayesian decision theory. In particular, we introduce a learnable scalar correction to the model's logits, optimized on a small validation set from the target distribution while keeping the backbone frozen. This parametric adjustment compensates for distributional shift in model output, realigning the decision boundary even without requiring ground-truth labels. Experiments on challenging benchmarks show that our approach significantly improves robustness without retraining, offering a lightweight and principled solution for reliable and adaptive AI-generated image detection in the open world. Code is available at https://github.com/muliyangm/AIGI-Det-Calib.