Pruning Unsafe Tickets: A Resource-Efficient Framework for Safer and More Robust LLMs

TL;DR

This paper presents a resource-efficient pruning method that identifies and removes unsafe behaviors in large language models, improving safety and robustness with minimal utility loss.

Contribution

It introduces a gradient-free attribution-based pruning framework that effectively reduces unsafe outputs in LLMs, revealing 'safety tickets' while preserving performance.

Findings

Significant reduction in unsafe generations in ML models.

Enhanced robustness against jailbreak attacks.

Minimal utility loss after pruning.

Abstract

Machine learning models are increasingly deployed in real-world applications, but even aligned models such as Mistral and LLaVA still exhibit unsafe behaviors inherited from pre-training. Current alignment methods like SFT and RLHF primarily encourage models to generate preferred responses, but do not explicitly remove the unsafe subnetworks that trigger harmful outputs. In this work, we introduce a resource-efficient pruning framework that directly identifies and removes parameters associated with unsafe behaviors while preserving model utility. Our method employs a gradient-free attribution mechanism, requiring only modest GPU resources, and generalizes across architectures and quantized variants. Empirical evaluations on ML models show substantial reductions in unsafe generations and improved robustness against jailbreak attacks, with minimal utility loss. From the perspective of the Lottery Ticket Hypothesis, our results suggest that ML models contain "unsafe tickets" responsible for harmful behaviors, and pruning reveals "safety tickets" that maintain performance while aligning outputs. This provides a lightweight, post-hoc alignment strategy suitable for deployment in resource-constrained settings.