Guarding the Meaning: Self-Supervised Training for Semantic Robustness in Guard Models

TL;DR

This paper introduces a self-supervised training framework that enhances the semantic robustness of guard models in large language models, reducing their sensitivity to superficial paraphrases and improving safety consistency.

Contribution

It proposes a skew-aware aggregation method for training guard models, significantly improving their semantic stability and calibration, and demonstrating its effectiveness on open-source models.

Findings

Reduces semantic variability across paraphrases by ~58%

Improves benchmark accuracy by ~2.5% on average

Enhances model calibration by up to 40%

Abstract

Guard models are a critical component of LLM safety, but their sensitivity to superficial linguistic variations remains a key vulnerability. We show that even meaning-preserving paraphrases can cause large fluctuations in safety scores, revealing a lack of semantic grounding. To address this, we introduce a practical, self-supervised framework for improving the semantic robustness of guard models. Our method leverages paraphrase sets to enforce prediction consistency using a novel, skew-aware aggregation strategy for robust target computation. Notably, we find that standard aggregation methods like mean and median can degrade safety, underscoring the need for skew-aware alternatives. We analyze six open-source guard models and show that our approach reduces semantic variability across paraphrases by ~58%, improves benchmark accuracy by ~2.5% on average, and generalizes to unseen stylistic variations. Intriguingly, we discover a bidirectional relationship between model calibration and consistency: our robustness training improves calibration by up to 40%, revealing a fundamental connection between these properties. These results highlight the value of treating semantic consistency as a first-class training objective and provide a scalable recipe for building more reliable guard models.