# Deep Mixture of Diverse Experts for Large-Scale Visual Recognition

## TL;DR

This paper introduces a deep mixture of diverse experts algorithm that combines multiple CNNs trained on different task groups for large-scale visual recognition, improving accuracy on tens of thousands of classes.

## Contribution

The paper proposes a novel deep mixture of experts framework with a hierarchical task grouping and multi-task learning to enhance large-scale image classification performance.

## Key findings

- Achieved competitive results on large-scale visual recognition datasets.
- Effectively grouped semantically related classes for improved learning.
- Leveraged inter-class similarities to enhance discriminative power.

## Abstract

In this paper, a deep mixture of diverse experts algorithm is developed for seamlessly combining a set of base deep CNNs (convolutional neural networks) with diverse outputs (task spaces), e.g., such base deep CNNs are trained to recognize different subsets of tens of thousands of atomic object classes. First, a two-layer (category layer and object class layer) ontology is constructed to achieve more effective solution for task group generation, e.g., assigning the semantically-related atomic object classes at the sibling leaf nodes into the same task group because they may share similar learning complexities. Second, one particular base deep CNNs with $M+1$ ($M \leq 1,000$) outputs is learned for each task group to recognize its $M$ atomic object classes effectively and identify one special class of "not-in-group" automatically, and the network structure (numbers of layers and units in each layer) of the well-designed AlexNet is directly used to configure such base deep CNNs. A deep multi-task learning algorithm is developed to leverage the inter-class visual similarities to learn more discriminative base deep CNNs and multi-task softmax for enhancing the separability of the atomic object classes in the same task group. Finally, all these base deep CNNs with diverse outputs (task spaces) are seamlessly combined to form a deep mixture of diverse experts for recognizing tens of thousands of atomic object classes. Our experimental results have demonstrated that our deep mixture of diverse experts algorithm can achieve very competitive results on large-scale visual recognition.