AutoBSS: An Efficient Algorithm for Block Stacking Style Search

TL;DR

AutoBSS introduces an efficient Bayesian optimization-based algorithm to automatically search for optimal block stacking styles in neural networks, significantly improving performance across multiple tasks.

Contribution

The paper presents AutoBSS, a novel AutoML method that effectively searches for block stacking styles, highlighting their importance in neural network design.

Findings

Achieves higher accuracy on ImageNet with searched BSS

Demonstrates strong generalizability in model compression and detection tasks

Outperforms baseline architectures without bias in evaluation

Abstract

Neural network architecture design mostly focuses on the new convolutional operator or special topological structure of network block, little attention is drawn to the configuration of stacking each block, called Block Stacking Style (BSS). Recent studies show that BSS may also have an unneglectable impact on networks, thus we design an efficient algorithm to search it automatically. The proposed method, AutoBSS, is a novel AutoML algorithm based on Bayesian optimization by iteratively refining and clustering Block Stacking Style Code (BSSC), which can find optimal BSS in a few trials without biased evaluation. On ImageNet classification task, ResNet50/MobileNetV2/EfficientNet-B0 with our searched BSS achieve 79.29%/74.5%/77.79%, which outperform the original baselines by a large margin. More importantly, experimental results on model compression, object detection and instance segmentation show the strong generalizability of the proposed AutoBSS, and further verify the unneglectable impact of BSS on neural networks.