Mixed Link Networks

TL;DR

This paper introduces MixNet, a flexible neural network architecture that combines the advantages of ResNet and DenseNet through modular inner and outer links, achieving superior efficiency on multiple datasets.

Contribution

The paper proposes MixNet, a novel network that unifies ResNet, DenseNet, and DPN, offering a modular design that improves efficiency and representation learning.

Findings

MixNet outperforms state-of-the-art architectures on CIFAR-10/100, SVHN, and ImageNet.

MixNet unifies ResNet, DenseNet, and DPN as special cases.

MixNet demonstrates superior parameter efficiency.

Abstract

Basing on the analysis by revealing the equivalence of modern networks, we find that both ResNet and DenseNet are essentially derived from the same "dense topology", yet they only differ in the form of connection -- addition (dubbed "inner link") vs. concatenation (dubbed "outer link"). However, both two forms of connections have the superiority and insufficiency. To combine their advantages and avoid certain limitations on representation learning, we present a highly efficient and modularized Mixed Link Network (MixNet) which is equipped with flexible inner link and outer link modules. Consequently, ResNet, DenseNet and Dual Path Network (DPN) can be regarded as a special case of MixNet, respectively. Furthermore, we demonstrate that MixNets can achieve superior efficiency in parameter over the state-of-the-art architectures on many competitive datasets like CIFAR-10/100, SVHN and ImageNet.