$n$-Reference Transfer Learning for Saliency Prediction

TL;DR

This paper introduces a few-shot transfer learning approach for saliency prediction that effectively adapts models trained on large datasets to new domains with limited data, improving performance significantly.

Contribution

It proposes a gradient-based, model-agnostic few-shot transfer learning framework specifically designed for saliency prediction tasks.

Findings

Significant performance improvements on various domain pairs.

Effective transfer of knowledge with very few target domain examples.

Framework is gradient-based and model-agnostic.

Abstract

Benefiting from deep learning research and large-scale datasets, saliency prediction has achieved significant success in the past decade. However, it still remains challenging to predict saliency maps on images in new domains that lack sufficient data for data-hungry models. To solve this problem, we propose a few-shot transfer learning paradigm for saliency prediction, which enables efficient transfer of knowledge learned from the existing large-scale saliency datasets to a target domain with limited labeled examples. Specifically, very few target domain examples are used as the reference to train a model with a source domain dataset such that the training process can converge to a local minimum in favor of the target domain. Then, the learned model is further fine-tuned with the reference. The proposed framework is gradient-based and model-agnostic. We conduct comprehensive experiments and ablation study on various source domain and target domain pairs. The results show that the proposed framework achieves a significant performance improvement. The code is publicly available at \url{https://github.com/luoyan407/n-reference}.