ParaNet - Using Dense Blocks for Early Inference

TL;DR

ParaNet is a novel architecture that enhances DenseNets by enabling early inference through multiple pipelines, parameter sharing, and logit matching, aiming to improve practicality and efficiency.

Contribution

Introduces ParaNet, a new architecture with multiple pipelines, parameter sharing, and logit matching to improve DenseNet's practicality for early inference.

Findings

Effective early inference with multiple pipelines

Parameter sharing reduces model complexity

Competitive performance on CIFAR-100

Abstract

DenseNets have been shown to be a competitive model among recent convolutional network architectures. These networks utilize Dense Blocks, which are groups of densely connected layers where the output of a hidden layer is fed in as the input of every other layer following it. In this paper, we aim to improve certain aspects of DenseNet, especially when it comes to practicality. We introduce ParaNet, a new architecture that constructs three pipelines which allow for early inference. We additionally introduce a cascading mechanism such that different pipelines are able to share parameters, as well as logit matching between the outputs of the pipelines. We separately evaluate each of the newly introduced mechanisms of ParaNet, then evaluate our proposed architecture on CIFAR-100.