Let the Experts Speak: Improving Survival Prediction & Calibration via Mixture-of-Experts Heads

TL;DR

This paper introduces advanced mixture-of-experts models for survival analysis that effectively cluster patients while improving calibration and predictive accuracy, emphasizing the importance of expert expressiveness.

Contribution

The work proposes novel discrete-time deep mixture-of-experts architectures that balance clustering, calibration, and accuracy, highlighting the role of expert expressiveness in performance.

Findings

More expressive experts outperform fixed prototypes.

One architecture achieves clustering, calibration, and accuracy simultaneously.

Expert expressiveness is crucial for model performance.

Abstract

Deep mixture-of-experts models have attracted a lot of attention for survival analysis problems, particularly for their ability to cluster similar patients together. In practice, grouping often comes at the expense of key metrics such as calibration error and predictive accuracy. This is due to the restrictive inductive bias that mixture-of-experts imposes, that predictions for individual patients must look like predictions for the group they're assigned to. Might we be able to discover patient group structure, where it exists, while improving calibration and predictive accuracy? In this work, we introduce several discrete-time deep mixture-of-experts (MoE)-based architectures for survival analysis problems, one of which achieves all desiderata: clustering, calibration, and predictive accuracy. We show that a key differentiator between this array of MoEs is how expressive their experts are. We find that more expressive experts that tailor predictions per patient outperform experts that rely on fixed group prototypes.