Beyond Model Base Retrieval: Weaving Knowledge to Master Fine-grained Neural Network Design

TL;DR

This paper introduces M-DESIGN, a retrieval-augmented framework that dynamically weaves historical architectural evidence to efficiently discover high-performing neural network modifications, outperforming existing methods.

Contribution

It proposes a novel evidence weaving approach with adaptive retrieval and predictive task planning to improve neural architecture search efficiency and effectiveness.

Findings

M-DESIGN outperforms baselines in 26 out of 33 cases.

Achieves near-optimal performance within strict search budgets.

Builds a knowledge base of 67,760 GNNs across 22 datasets.

Abstract

Designing high-performance neural networks for new tasks requires balancing optimization quality with search efficiency. Current methods fail to achieve this balance: neural architectural search is computationally expensive, while model retrieval often yields suboptimal static checkpoints. To resolve this dilemma, we model the performance gains induced by fine-grained architectural modifications as edit-effect evidence and build evidence graphs from prior tasks. By constructing a retrieval-augmented model refinement framework, our proposed M-DESIGN dynamically weaves historical evidence to discover near-optimal modification paths. M-DESIGN features an adaptive retrieval mechanism that quickly calibrates the evolving transferability of edit-effect evidence from different sources. To handle out-of-distribution shifts, we introduce predictive task planners that extrapolate gains from multi-hop evidence, thereby reducing reliance on an exhaustive repository. Based on our model knowledge base of 67,760 graph neural networks across 22 datasets, extensive experiments demonstrate that M-DESIGN consistently outperforms baselines, achieving the search-space best performance in 26 out of 33 cases under a strict budget.