Evolutionary Architecture Search through Grammar-Based Sequence Alignment
Adri G\'omez Mart\'in, Felix M\"oller, Steven McDonagh, Monica Abella, Manuel Desco, Elliot J. Crowley, Aaron Klein, Linus Ericsson
TL;DR
This paper introduces a novel method using adapted Smith-Waterman algorithms for efficient sequence alignment in grammar-based neural architecture search, enabling better exploration and hybridization of architectures.
Contribution
It adapts Smith-Waterman for neural architecture search, improving computational efficiency and enabling effective crossover operations in evolutionary algorithms.
Findings
Outperforms existing NAS methods in search efficiency
Enables computation of shortest paths between architectures
Facilitates analysis of architectural loss landscape
Abstract
Neural architecture search (NAS) in expressive search spaces is a computationally hard problem, but it also holds the potential to automatically discover completely novel and performant architectures. To achieve this we need effective search algorithms that can identify powerful components and reuse them in new candidate architectures. In this paper, we introduce two adapted variants of the Smith-Waterman algorithm for local sequence alignment and use them to compute the edit distance in a grammar-based evolutionary architecture search. These algorithms enable us to efficiently calculate a distance metric for neural architectures and to generate a set of hybrid offspring from two parent models. This facilitates the deployment of crossover-based search heuristics, allows us to perform a thorough analysis on the architectural loss landscape, and track population diversity during search.…
Peer Reviews
Decision·Submitted to ICLR 2026
- The proposed variants seem reasonable. - As shown in Table 1, crossover shows better performance than mutation only.
- As stated in the conclusion, CSWX sometimes outperforms RCSWX, which requires a deep investigation. - In Figure 2, no crossover shows better validation performance on Chesseract and Isabella, but achieves lower test performance on test set as shown in Table 1. It is unclear why this happens. - The performance gain is not consistent across datasets, which raises concerns about the stability of the method when applied to different datasets. Sometimes CSWX does not better than STX (see Chesserac
1. (R)CSWX is an interesting algorithm, building on the Smith-Waterman algorithm, for NAS. 2. Runtime analysis presents good upper bound computational complexity.
W1. RCSWX which is the more tractable and practical algorithm underperforms the baselines. W2. The authors offer insufficient analysis or explanation for W1. W3. The authors state that the focus of this work is to "introduce a theoretically sound, computationally efficient crossover operator for grammar-based NAS, intended as a tool for further research rather than as a benchmark for state-of-the-art performance." but this is not a sufficient reason to not compare CSWX and RCSWX with other opti
+ The paper clearly shows that prior methods based on Graph Edit Distance (e.g., SEPX) are NP-hard and become intractable for even moderately sized graphs. The proposed (R)CSWX methods, by contrast, are highly efficient, effectively scaling to large architectures. This is a significant practical contribution. + The proposed method is valuable as both a search operator and an analysis tool. Using $d_{RCSWX}$ to perform a large-scale analysis of the architectural loss landscape may be a compellin
+ As is well known, the primary computational cost of NAS is the performance estimation (i.e., architecture evaluation), not the search strategy. Therefore, the main contribution of this paper, which enhances the efficiency of the search strategy, seems less significant in the context of the overall NAS pipeline. + In Table 1, RCSWX shows the lowest average performance, even underperforming the "No Crossover" baseline. Its result on AddNIST, in particular, is very poor. This undermines the cent
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsEvolutionary Algorithms and Applications · Metaheuristic Optimization Algorithms Research · Advanced Software Engineering Methodologies