What Matters For Meta-Learning Vision Regression Tasks?

TL;DR

This paper explores meta-learning for high-dimensional vision regression tasks, introduces new challenging benchmarks, evaluates existing techniques, and proposes functional contrastive learning to improve task representation in CNPs.

Contribution

It designs complex vision regression benchmarks, analyzes meta-learning techniques and deep learning strategies, and introduces functional contrastive learning for better task representations in CNPs.

Findings

CNPs outperform MAML on most tasks without fine-tuning

Meta-training set size affects results significantly

Naive task augmentation can cause underfitting

Abstract

Meta-learning is widely used in few-shot classification and function regression due to its ability to quickly adapt to unseen tasks. However, it has not yet been well explored on regression tasks with high dimensional inputs such as images. This paper makes two main contributions that help understand this barely explored area. \emph{First}, we design two new types of cross-category level vision regression tasks, namely object discovery and pose estimation of unprecedented complexity in the meta-learning domain for computer vision. To this end, we (i) exhaustively evaluate common meta-learning techniques on these tasks, and (ii) quantitatively analyze the effect of various deep learning techniques commonly used in recent meta-learning algorithms in order to strengthen the generalization capability: data augmentation, domain randomization, task augmentation and meta-regularization. Finally, we (iii) provide some insights and practical recommendations for training meta-learning algorithms on vision regression tasks. \emph{Second}, we propose the addition of functional contrastive learning (FCL) over the task representations in Conditional Neural Processes (CNPs) and train in an end-to-end fashion. The experimental results show that the results of prior work are misleading as a consequence of a poor choice of the loss function as well as too small meta-training sets. Specifically, we find that CNPs outperform MAML on most tasks without fine-tuning. Furthermore, we observe that naive task augmentation without a tailored design results in underfitting.