Mixture of Predefined Experts: Maximizing Data Usage on Vertical Federated Learning

TL;DR

Split-MoPE is a novel VFL framework combining Split Learning with a predefined experts architecture, maximizing data utilization and robustness in real-world, misaligned sample scenarios.

Contribution

It introduces a new architecture that uses predefined experts for data alignment, enabling efficient, robust, and interpretable federated learning without full sample overlap.

Findings

Achieves state-of-the-art performance with a single communication round.

Reduces communication overhead compared to multi-round training.

Demonstrates robustness against malicious or noisy participants.

Abstract

Vertical Federated Learning (VFL) has emerged as a critical paradigm for collaborative model training in privacy-sensitive domains such as finance and healthcare. However, most existing VFL frameworks rely on the idealized assumption of full sample alignment across participants, a premise that rarely holds in real-world scenarios. To bridge this gap, this work introduces Split-MoPE, a novel framework that integrates Split Learning with a specialized Mixture of Predefined Experts (MoPE) architecture. Unlike standard Mixture of Experts (MoE), where routing is learned dynamically, MoPE uses predefined experts to process specific data alignments, effectively maximizing data usage during both training and inference without requiring full sample overlap. By leveraging pretrained encoders for target data domains, Split-MoPE achieves state-of-the-art performance in a single communication round, significantly reducing the communication footprint compared to multi-round end-to-end training. Furthermore, unlike existing proposals that address sample misalignment, this novel architecture provides inherent robustness against malicious or noisy participants and offers per-sample interpretability by quantifying each collaborator's contribution to each prediction. Extensive evaluations on vision (CIFAR-10/100) and tabular (Breast Cancer Wisconsin) datasets demonstrate that Split-MoPE consistently outperforms state-of-the-art systems such as LASER and Vertical SplitNN, particularly in challenging scenarios with high data missingness.