MENTOR: A Metacognition-Driven Self-Evolution Framework for Uncovering and Mitigating Implicit Domain Risks in LLMs

TL;DR

This paper introduces MENTOR, a framework that uses metacognition and self-evolution to identify and mitigate implicit domain risks in LLMs, significantly improving safety across various sectors.

Contribution

The paper presents a novel metacognition-driven framework that dynamically detects and reduces domain-specific risks in LLMs through self-assessment and rule-based knowledge evolution.

Findings

MENTOR reduces jailbreak success rates across multiple domains.

It achieves risk analysis performance comparable to human experts.

The framework effectively enhances LLM safety and robustness.

Abstract

Ensuring the safety of Large Language Models (LLMs) is critical for real-world deployment. However, current safety measures often fail to address implicit, domain-specific risks. To investigate this gap, we introduce a dataset of 3,000 annotated queries spanning education, finance, and management. Evaluations across 14 leading LLMs reveal a concerning vulnerability: an average jailbreak success rate of 57.8%. In response, we propose MENTOR, a metacognition-driven self-evolution framework. MENTOR first performs structured self-assessment through simulated critical thinking, such as perspective-taking and consequential reasoning to uncover latent model misalignments. These reflections are formalized into dynamic rule-based knowledge graphs that evolve with emerging risk patterns. To enforce these rules at inference time, we introduce activation steering, a method that directly modulates the model's internal representations to ensure compliance. Experiments demonstrate that MENTOR substantially reduces attack success rates across all tested domains and achieves risk analysis performance comparable to human experts. Our work offers a scalable and adaptive pathway toward robust domain-specific alignment of LLMs.