HHNAS-AM: Hierarchical Hybrid Neural Architecture Search using Adaptive Mutation Policies

TL;DR

HHNAS-AM introduces a hierarchical, adaptive mutation-based neural architecture search method that efficiently explores structured search spaces, leading to improved text classification performance.

Contribution

It proposes a novel hierarchical hybrid NAS framework with adaptive mutation policies guided by Q-learning, enhancing search efficiency and architecture quality.

Findings

Achieves 8% higher accuracy on Spider dataset.

Effectively explores structured search spaces with adaptive mutations.

Outperforms existing NAS methods in text classification tasks.

Abstract

Neural Architecture Search (NAS) has garnered significant research interest due to its capability to discover architectures superior to manually designed ones. Learning text representation is crucial for text classification and other language-related tasks. The NAS model used in text classification does not have a Hybrid hierarchical structure, and there is no restriction on the architecture structure, due to which the search space becomes very large and mostly redundant, so the existing RL models are not able to navigate the search space effectively. Also, doing a flat architecture search leads to an unorganised search space, which is difficult to traverse. For this purpose, we propose HHNAS-AM (Hierarchical Hybrid Neural Architecture Search with Adaptive Mutation Policies), a novel approach that efficiently explores diverse architectural configurations. We introduce a few architectural templates to search on which organise the search spaces, where search spaces are designed on the basis of domain-specific cues. Our method employs mutation strategies that dynamically adapt based on performance feedback from previous iterations using Q-learning, enabling a more effective and accelerated traversal of the search space. The proposed model is fully probabilistic, enabling effective exploration of the search space. We evaluate our approach on the database id (db_id) prediction task, where it consistently discovers high-performing architectures across multiple experiments. On the Spider dataset, our method achieves an 8% improvement in test accuracy over existing baselines.