A Multi-objective Evolutionary Algorithm Based on Bi-population with Uniform Sampling for Neural Architecture Search

TL;DR

This paper introduces MOEA-BUS, a multi-objective evolutionary algorithm with bi-population and uniform sampling, to efficiently discover neural network architectures optimizing accuracy and complexity, outperforming existing methods.

Contribution

It proposes a novel bi-population framework with uniform sampling for neural architecture search, improving diversity and exploration over prior approaches.

Findings

Achieved 98.39% accuracy on CIFAR-10

Reached 80.03% top-1 accuracy on ImageNet

Enhanced population diversity and search performance

Abstract

Neural architecture search (NAS) automates neural network design, improving efficiency over manual approaches. However, efficiently discovering high-performance neural network architectures that simultaneously optimize multiple objectives remains a significant challenge in NAS. Existing methods often suffer from limited population diversity and inadequate exploration of the search space, particularly in regions with extreme complexity values. To address these challenges, we propose MOEA-BUS, an innovative multi-objective evolutionary algorithm based on bi-population with uniform sampling for neural architecture search, aimed at simultaneously optimizing both accuracy and network complexity. In MOEA-BUS, a novel uniform sampling method is proposed to initialize the population, ensuring that architectures are distributed uniformly across the objective space. Furthermore, to enhance exploration, we deploy a bi-population framework where two populations evolve synergistically, facilitating comprehensive search space coverage. Experiments on CIFAR-10 and ImageNet demonstrate MOEA-BUS's superiority, achieving top-1 accuracies of 98.39% on CIFAR-10, and 80.03% on ImageNet. Notably, it achieves 78.28% accuracy on ImageNet with only 446M MAdds. Ablation studies confirm that both uniform sampling and bi-population mechanisms enhance population diversity and performance. Additionally, in terms of the Kendall's tau coefficient, the SVM achieves an improvement of at least 0.035 compared to the other three commonly used machine learning models, and uniform sampling provided an enhancement of approximately 0.07.