Hierarchical Routing Mixture of Experts
Wenbo Zhao, Yang Gao, Shahan Ali Memon, Bhiksha Raj, Rita Singh

TL;DR
This paper introduces a hierarchical routing mixture of experts model that uses a binary tree structure with classifiers and regressors to better handle complex, multimodal regression data by jointly partitioning the input-output space.
Contribution
The paper proposes a novel binary tree-structured HRME model with a probabilistic framework and EM algorithm for learning, improving regression on complex multimodal data.
Findings
Outperforms other regression models on various tasks.
Effectively partitions complex data with classifiers and simple regressors.
Demonstrates the benefit of hierarchical routing in regression accuracy.
Abstract
In regression tasks the distribution of the data is often too complex to be fitted by a single model. In contrast, partition-based models are developed where data is divided and fitted by local models. These models partition the input space and do not leverage the input-output dependency of multimodal-distributed data, and strong local models are needed to make good predictions. Addressing these problems, we propose a binary tree-structured hierarchical routing mixture of experts (HRME) model that has classifiers as non-leaf node experts and simple regression models as leaf node experts. The classifier nodes jointly soft-partition the input-output space based on the natural separateness of multimodal data. This enables simple leaf experts to be effective for prediction. Further, we develop a probabilistic framework for the HRME model, and propose a recursive Expectation-Maximization…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNeural Networks and Applications · Data Stream Mining Techniques · Gaussian Processes and Bayesian Inference
