$\mu$DARTS: Model Uncertainty-Aware Differentiable Architecture Search
Biswadeep Chakraborty, Saibal Mukhopadhyay

TL;DR
$DARTS is a neural architecture search method that incorporates model uncertainty optimization using concrete dropout and a Monte-Carlo regularizer, resulting in architectures with higher accuracy, lower uncertainty, and increased robustness.
Contribution
The paper introduces $DARTS, a novel differentiable architecture search method that explicitly optimizes for both accuracy and model uncertainty using concrete dropout and a new regularizer.
Findings
$DARTS improves accuracy on CIFAR10, CIFAR100, SVHN, and ImageNet.
It reduces model uncertainty compared to existing DARTS methods.
The resulting architectures are more robust to noise.
Abstract
We present a Model Uncertainty-aware Differentiable ARchiTecture Search (DARTS) that optimizes neural networks to simultaneously achieve high accuracy and low uncertainty. We introduce concrete dropout within DARTS cells and include a Monte-Carlo regularizer within the training loss to optimize the concrete dropout probabilities. A predictive variance term is introduced in the validation loss to enable searching for architecture with minimal model uncertainty. The experiments on CIFAR10, CIFAR100, SVHN, and ImageNet verify the effectiveness of DARTS in improving accuracy and reducing uncertainty compared to existing DARTS methods. Moreover, the final architecture obtained from DARTS shows higher robustness to noise at the input image and model parameters compared to the architecture obtained from existing DARTS methods.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsService-Oriented Architecture and Web Services · Advanced Database Systems and Queries · Business Process Modeling and Analysis
MethodsDifferentiable Architecture Search · Concrete Dropout · Dropout
