Latent Personality Alignment: Improving Harmlessness Without Mentioning Harms

TL;DR

Latent Personality Alignment (LPA) enhances language model harmlessness by training on abstract traits, achieving robustness with fewer data and better generalization to unseen attacks.

Contribution

LPA introduces a personality-based training method that requires minimal data and improves robustness against novel harmful prompts without using harmful examples.

Findings

LPA matches robustness of large-data methods with fewer than 100 trait statements.

LPA reduces misclassification rates by 2.6x across six harm benchmarks.

LPA generalizes better to unseen attack distributions.

Abstract

Current adversarial robustness methods for large language models require extensive datasets of harmful prompts (thousands to hundreds of thousands of examples), yet remain vulnerable to novel attack vectors and distributional shifts. We propose Latent Personality Alignment (LPA), a sample-efficient defense that achieves robustness by training models on abstract personality traits rather than specific harmful behaviors. Using fewer than 100 trait statements and latent adversarial training, LPA achieves comparable attack success rates to methods trained on 150k+ examples, while maintaining superior utility. Critically, LPA generalizes better to unseen attack distributions, reducing misclassification rates by 2.6x compared to baseline across six harm benchmarks -- without ever seeing harmful examples during training. Our results demonstrate that personality-based alignment offers a principled approach to building robust defenses with minimal cost.