ConstitutionalExperts: Training a Mixture of Principle-based Prompts

TL;DR

ConstitutionalExperts introduces a novel approach for training prompts based on constitutional principles, incrementally refining them and employing a mixture-of-experts architecture to enhance large language model performance across diverse tasks.

Contribution

The paper presents a new method for learning and editing principle-based prompts and demonstrates the effectiveness of a mixture-of-experts architecture for improved prompt routing and performance.

Findings

Outperforms existing prompt-optimization methods by 10.9% F1 score.

Mixture-of-experts architecture enhances performance across techniques.

Incremental prompt editing improves task-specific results.

Abstract

Large language models (LLMs) are highly capable at a variety of tasks given the right prompt, but writing one is still a difficult and tedious process. In this work, we introduce ConstitutionalExperts, a method for learning a prompt consisting of constitutional principles (i.e. rules), given a training dataset. Unlike prior methods that optimize the prompt as a single entity, our method incrementally improves the prompt by surgically editing individual principles. We also show that we can improve overall performance by learning unique prompts for different semantic regions of the training data and using a mixture-of-experts (MoE) architecture to route inputs at inference time. We compare our method to other state of the art prompt-optimization techniques across six benchmark datasets. We also investigate whether MoE improves these other techniques. Our results suggest that ConstitutionalExperts outperforms other prompt optimization techniques by 10.9% (F1) and that mixture-of-experts improves all techniques, suggesting its broad applicability.