HERCULES: Hardware-Efficient, Robust, Continual Learning Neural Architecture Search

TL;DR

HERCULES introduces a comprehensive framework for neural architecture search that balances efficiency, robustness, and continual learning to meet real-world deployment needs.

Contribution

It proposes a new taxonomy and framework for multi-objective NAS focusing on resource efficiency, environmental resilience, and architectural plasticity.

Findings

Mapped current NAS methods into the HERCULES framework.

Identified gaps in existing research for integrated multi-objective NAS.

Outlined a roadmap for future research in deployable lifelong-learning AI.

Abstract

Neural Architecture Search (NAS) has emerged as a powerful framework for automatically discovering neural architectures that balance accuracy and efficiency. However, as AI transitions from static benchmarks to real-world deployment, the traditional focus on hardware-aware efficiency is no longer sufficient. We observe that modern NAS methods, especially those that target edge AI, are evolving to address a triple objective: Efficiency, Robustness, and Continual Learning. While efficiency ensures feasibility in resource-constrained environments, robustness guarantees reliability under environmental variabilities, and continual learning enables adaptation to sequential tasks without catastrophic forgetting. We propose a taxonomy of NAS approaches through this triple lens, distinguishing between methods targeting resource optimization, environmental resilience, and architectural plasticity. This unified perspective reveals that these axes, though often studied in isolation, are mutually reinforcing. Building on this taxonomy, we map the current landscape of these NAS methods into a new framework called Hardware-Efficient, Robust, and ContinUal LEarning Search (HERCULES). We define the desiderata, the twelve labours of HERCULES, addressing the non-trivial challenge of balancing an adequate search-space exploration with the immense computational costs of a multi-objective NAS, accounting for these crucial objectives of current AI systems. By identifying critical gaps in existing research, this survey outlines a roadmap toward integrated algorithmic, architectural, and hardware-software co-design for truly deployable, lifelong-learning AI systems.