$\mu$DARTS: Model Uncertainty-Aware Differentiable Architecture Search

TL;DR

$DARTS is a neural architecture search method that incorporates model uncertainty optimization using concrete dropout and a Monte-Carlo regularizer, resulting in architectures with higher accuracy, lower uncertainty, and increased robustness.

Contribution

The paper introduces $DARTS, a novel differentiable architecture search method that explicitly optimizes for both accuracy and model uncertainty using concrete dropout and a new regularizer.

Findings

$DARTS improves accuracy on CIFAR10, CIFAR100, SVHN, and ImageNet.

It reduces model uncertainty compared to existing DARTS methods.

The resulting architectures are more robust to noise.

Abstract

We present a Model Uncertainty-aware Differentiable ARchiTecture Search ($\mu$DARTS) that optimizes neural networks to simultaneously achieve high accuracy and low uncertainty. We introduce concrete dropout within DARTS cells and include a Monte-Carlo regularizer within the training loss to optimize the concrete dropout probabilities. A predictive variance term is introduced in the validation loss to enable searching for architecture with minimal model uncertainty. The experiments on CIFAR10, CIFAR100, SVHN, and ImageNet verify the effectiveness of $\mu$DARTS in improving accuracy and reducing uncertainty compared to existing DARTS methods. Moreover, the final architecture obtained from $\mu$DARTS shows higher robustness to noise at the input image and model parameters compared to the architecture obtained from existing DARTS methods.