Deep Elastic Networks with Model Selection for Multi-Task Learning

TL;DR

This paper introduces a dynamic model selection method for multi-task learning that adaptively chooses the most suitable network configuration for each instance, improving accuracy and efficiency.

Contribution

It proposes a unified framework with an estimator and selector for instance-wise model selection in multi-task learning, enhancing flexibility and performance.

Findings

Outperforms existing multi-task learning approaches in accuracy.

Enables efficient instance-wise model selection without extra computation.

Demonstrates versatility across multiple image classification tasks.

Abstract

In this work, we consider the problem of instance-wise dynamic network model selection for multi-task learning. To this end, we propose an efficient approach to exploit a compact but accurate model in a backbone architecture for each instance of all tasks. The proposed method consists of an estimator and a selector. The estimator is based on a backbone architecture and structured hierarchically. It can produce multiple different network models of different configurations in a hierarchical structure. The selector chooses a model dynamically from a pool of candidate models given an input instance. The selector is a relatively small-size network consisting of a few layers, which estimates a probability distribution over the candidate models when an input instance of a task is given. Both estimator and selector are jointly trained in a unified learning framework in conjunction with a sampling-based learning strategy, without additional computation steps. We demonstrate the proposed approach for several image classification tasks compared to existing approaches performing model selection or learning multiple tasks. Experimental results show that our approach gives not only outstanding performance compared to other competitors but also the versatility to perform instance-wise model selection for multiple tasks.