Efficient Task Grouping Through Samplewise Optimisation Landscape Analysis
Anshul Thakur, Yichen Huang, Soheila Molaei, Yujiang Wang, and David, A. Clifton

TL;DR
This paper presents a novel, efficient framework for task grouping in multi-task learning that infers task similarities through samplewise landscape analysis, significantly reducing computational costs while maintaining high performance.
Contribution
It introduces a samplewise optimisation landscape analysis method to infer task similarities, enabling faster and effective task grouping without extensive shared model training.
Findings
Five-fold speed improvement over state-of-the-art methods
Consistent task grouping performance across 8 datasets
Effective identification of near-optimal task groups
Abstract
Shared training approaches, such as multi-task learning (MTL) and gradient-based meta-learning, are widely used in various machine learning applications, but they often suffer from negative transfer, leading to performance degradation in specific tasks. While several optimisation techniques have been developed to mitigate this issue for pre-selected task cohorts, identifying optimal task combinations for joint learning - known as task grouping - remains underexplored and computationally challenging due to the exponential growth in task combinations and the need for extensive training and evaluation cycles. This paper introduces an efficient task grouping framework designed to reduce these overwhelming computational demands of the existing methods. The proposed framework infers pairwise task similarities through a sample-wise optimisation landscape analysis, eliminating the need for the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSimulation Techniques and Applications
MethodsSPEED: Separable Pyramidal Pooling EncodEr-Decoder for Real-Time Monocular Depth Estimation on Low-Resource Settings
