Exploiting the Experts: Unauthorized Compression in MoE-LLMs

TL;DR

This paper investigates the vulnerabilities of MoE-LLMs to unauthorized model compression and fine-tuning, proposing methods to identify, evaluate, and defend against such exploits to enhance model security.

Contribution

It introduces an expert attribution framework and evaluates the impact of pruning on task performance, proposing defense strategies to prevent unauthorized model adaptation.

Findings

Expert pruning can significantly degrade model performance without proper re-alignment.

Targeted fine-tuning can recover task accuracy after pruning, revealing a knowledge loss-recovery trade-off.

Proposed defense strategies increase resistance to unauthorized compression and fine-tuning.

Abstract

Mixture-of-Experts (MoE) architectures are increasingly adopted in large language models (LLMs) for their scalability and efficiency. However, their modular structure introduces a unique vulnerability: adversaries can attempt to compress or repurpose models by pruning experts and cheaply fine-tuning the remainder, effectively bypassing licensing and security constraints. In this paper, we systematically study the prunability of MoE-LLMs under task-specific usage. We first develop an expert attribution framework that identifies the subset of experts most responsible for a given task, then evaluate the performance trade-offs of pruning and re-aligning these experts using active learning-driven fine-tuning. Our findings reveal a critical knowledge loss--recovery trade-off: while certain experts can be isolated to retain task accuracy, significant degradation occurs without targeted re-alignment. Based on this analysis, we propose defense strategies that aim to make MoE models harder to compress and fine-tune without authorization, including entangled expert training and selective fine-tuning protocols that resist unauthorized adaptation. By positioning expert pruning as both a threat vector and a defense target, this work highlights the dual-use nature of MoE modularity and provides the first systematic evaluation framework for secure specialization of MoE-LLMs.