Universal Hypernetworks for Arbitrary Models

TL;DR

The paper introduces Universal Hypernetworks (UHNs), a fixed-architecture generator capable of producing weights for diverse models across multiple domains and tasks, enabling flexible, multi-model, and multi-task learning.

Contribution

It proposes a descriptor-based universal hypernetwork that decouples generator architecture from target model parameterization, supporting heterogeneous models and recursive generation.

Findings

UHN remains competitive with direct training across vision, graph, text, and formula regression tasks.

The same UHN supports multi-model generalization and multi-task learning.

UHN enables stable recursive generation with up to three intermediate UHNs.

Abstract

Conventional hypernetworks are typically engineered around a specific base-model parameterization, so changing the target architecture often entails redesigning the hypernetwork and retraining it from scratch. We introduce the \emph{Universal Hypernetwork} (UHN), a fixed-architecture generator that predicts weights from deterministic parameter, architecture, and task descriptors. This descriptor-based formulation decouples the generator architecture from target-network parameterization, so one generator can instantiate heterogeneous models across the tested architecture and task families. Our empirical claims are threefold: (1) one fixed UHN remains competitive with direct training across vision, graph, text, and formula-regression benchmarks; (2) the same UHN supports both multi-model generalization within a family and multi-task learning across heterogeneous models; and (3) UHN enables stable recursive generation with up to three intermediate generated UHNs before the final base model. Our code is available at https://github.com/Xuanfeng-Zhou/UHN.