Efficient LLM Moderation with Multi-Layer Latent Prototypes

TL;DR

The paper introduces MLPM, a lightweight, multi-layer prototype-based moderation tool that enhances safety and efficiency in deploying large language models, outperforming existing methods across various benchmarks.

Contribution

We propose a novel multi-layer prototype approach for LLM moderation that is highly customizable, efficient, and easily integrable into existing pipelines.

Findings

Achieves state-of-the-art moderation performance

Maintains high efficiency with negligible overhead

Scales effectively across different model sizes

Abstract

Although modern LLMs are aligned with human values during post-training, robust moderation remains essential to prevent harmful outputs at deployment time. Existing approaches suffer from performance-efficiency trade-offs and are difficult to customize to user-specific requirements. Motivated by this gap, we introduce Multi-Layer Prototype Moderator (MLPM), a lightweight and highly customizable input moderation tool. We propose leveraging prototypes of intermediate representations across multiple layers to improve moderation quality while maintaining high efficiency. By design, our method adds negligible overhead to the generation pipeline and can be seamlessly applied to any model. MLPM achieves state-of-the-art performance on diverse moderation benchmarks and demonstrates strong scalability across model families of various sizes. Moreover, we show that it integrates smoothly into end-to-end moderation pipelines and further improves response safety when combined with output moderation techniques. Overall, our work provides a practical and adaptable solution for safe, robust, and efficient LLM deployment.