SalNAS: Efficient Saliency-prediction Neural Architecture Search with self-knowledge distillation

TL;DR

SalNAS introduces an efficient neural architecture search framework for saliency prediction, utilizing a supernet with dynamic convolution and a self-knowledge distillation method to enhance generalization and outperform state-of-the-art models.

Contribution

The paper presents a novel NAS framework with a dynamic convolution supernet and a self-knowledge distillation approach, improving saliency prediction accuracy and efficiency.

Findings

Outperforms state-of-the-art models on seven benchmarks

Achieves high accuracy with only 20.98 million parameters

Efficient training without gradient computation in the teacher model

Abstract

Recent advancements in deep convolutional neural networks have significantly improved the performance of saliency prediction. However, the manual configuration of the neural network architectures requires domain knowledge expertise and can still be time-consuming and error-prone. To solve this, we propose a new Neural Architecture Search (NAS) framework for saliency prediction with two contributions. Firstly, a supernet for saliency prediction is built with a weight-sharing network containing all candidate architectures, by integrating a dynamic convolution into the encoder-decoder in the supernet, termed SalNAS. Secondly, despite the fact that SalNAS is highly efficient (20.98 million parameters), it can suffer from the lack of generalization. To solve this, we propose a self-knowledge distillation approach, termed Self-KD, that trains the student SalNAS with the weighted average information between the ground truth and the prediction from the teacher model. The teacher model, while sharing the same architecture, contains the best-performing weights chosen by cross-validation. Self-KD can generalize well without the need to compute the gradient in the teacher model, enabling an efficient training system. By utilizing Self-KD, SalNAS outperforms other state-of-the-art saliency prediction models in most evaluation rubrics across seven benchmark datasets while being a lightweight model. The code will be available at https://github.com/chakkritte/SalNAS