Acquiring and Adapting Priors for Novel Tasks via Neural Meta-Architectures

TL;DR

This paper introduces neural meta-architectures, including hypernetworks and memory modules, to efficiently acquire and adapt priors for novel tasks with limited data, demonstrating improvements in 3D scene generation, segmentation, and molecular property prediction.

Contribution

The work presents new neural architectures and training strategies that enable rapid transfer and adaptation of priors in data-scarce domains, outperforming standard methods.

Findings

Hypernetworks acquire more generalizable priors than standard networks.

Memory modules enable adaptation on non-stationary distributions with few samples.

Pre-training with molecular generative models improves property prediction.

Abstract

The ability to transfer knowledge from prior experiences to novel tasks stands as a pivotal capability of intelligent agents, including both humans and computational models. This principle forms the basis of transfer learning, where large pre-trained neural networks are fine-tuned to adapt to downstream tasks. Transfer learning has demonstrated tremendous success, both in terms of task adaptation speed and performance. However there are several domains where, due to lack of data, training such large pre-trained models or foundational models is not a possibility - computational chemistry, computational immunology, and medical imaging are examples. To address these challenges, our work focuses on designing architectures to enable efficient acquisition of priors when large amounts of data are unavailable. In particular, we demonstrate that we can use neural memory to enable adaptation on non-stationary distributions with only a few samples. Then we demonstrate that our hypernetwork designs (a network that generates another network) can acquire more generalizable priors than standard networks when trained with Model Agnostic Meta-Learning (MAML). Subsequently, we apply hypernetworks to 3D scene generation, demonstrating that they can acquire priors efficiently on just a handful of training scenes, thereby leading to faster text-to-3D generation. We then extend our hypernetwork framework to perform 3D segmentation on novel scenes with limited data by efficiently transferring priors from earlier viewed scenes. Finally, we repurpose an existing molecular generative method as a pre-training framework that facilitates improved molecular property prediction, addressing critical challenges in computational immunology.