Large Language Model Assisted Adversarial Robustness Neural Architecture Search

TL;DR

This paper introduces a novel LLM-assisted optimizer for adversarial robustness neural architecture search, demonstrating its competitiveness through experiments on NAS-Bench-201 datasets with CIFAR-10 and CIFAR-100.

Contribution

It proposes a new LLM-based approach for ARNAS, leveraging prompt refinement and LLM responses as solutions, showcasing the potential of LLMs as effective combinatorial optimizers.

Findings

LLMO outperforms traditional meta-heuristic algorithms in ARNAS tasks.

The approach demonstrates competitiveness on NAS-Bench-201 datasets.

Source code is publicly available for reproducibility.

Abstract

Motivated by the potential of large language models (LLMs) as optimizers for solving combinatorial optimization problems, this paper proposes a novel LLM-assisted optimizer (LLMO) to address adversarial robustness neural architecture search (ARNAS), a specific application of combinatorial optimization. We design the prompt using the standard CRISPE framework (i.e., Capacity and Role, Insight, Statement, Personality, and Experiment). In this study, we employ Gemini, a powerful LLM developed by Google. We iteratively refine the prompt, and the responses from Gemini are adapted as solutions to ARNAS instances. Numerical experiments are conducted on NAS-Bench-201-based ARNAS tasks with CIFAR-10 and CIFAR-100 datasets. Six well-known meta-heuristic algorithms (MHAs) including genetic algorithm (GA), particle swarm optimization (PSO), differential evolution (DE), and its variants serve as baselines. The experimental results confirm the competitiveness of the proposed LLMO and highlight the potential of LLMs as effective combinatorial optimizers. The source code of this research can be downloaded from \url{https://github.com/RuiZhong961230/LLMO}.