muNet: Evolving Pretrained Deep Neural Networks into Scalable Auto-tuning Multitask Systems

TL;DR

This paper introduces muNet, an evolutionary method that constructs scalable multitask neural networks from pretrained models, enabling efficient knowledge transfer, auto-tuning, and improved accuracy with fewer parameters across multiple tasks.

Contribution

The paper presents a novel evolutionary approach for building multitask neural networks from pretrained models, addressing knowledge transfer and hyperparameter auto-tuning.

Findings

Outperforms standard fine-tuning with 2.39% higher accuracy.

Uses 47% fewer parameters per task.

Effectively handles multiple diverse image classification tasks.

Abstract

Most uses of machine learning today involve training a model from scratch for a particular task, or sometimes starting with a model pretrained on a related task and then fine-tuning on a downstream task. Both approaches offer limited knowledge transfer between different tasks, time-consuming human-driven customization to individual tasks and high computational costs especially when starting from randomly initialized models. We propose a method that uses the layers of a pretrained deep neural network as building blocks to construct an ML system that can jointly solve an arbitrary number of tasks. The resulting system can leverage cross tasks knowledge transfer, while being immune from common drawbacks of multitask approaches such as catastrophic forgetting, gradients interference and negative transfer. We define an evolutionary approach designed to jointly select the prior knowledge relevant for each task, choose the subset of the model parameters to train and dynamically auto-tune its hyperparameters. Furthermore, a novel scale control method is employed to achieve quality/size trade-offs that outperform common fine-tuning techniques. Compared with standard fine-tuning on a benchmark of 10 diverse image classification tasks, the proposed model improves the average accuracy by 2.39% while using 47% less parameters per task.