Pseudo-Labeling by Multi-Policy Viewfinder Network for Image Cropping

TL;DR

This paper introduces MPV-Net, a multi-policy viewfinder network that enhances pseudo-labeling for image cropping by selecting the most reliable policy, leading to improved performance and state-of-the-art results.

Contribution

The paper proposes a novel multi-policy approach to refine pseudo labels in image cropping, addressing teacher mistakes and improving training with unlabeled data.

Findings

MPV-Net outperforms existing pseudo-labeling methods.

Achieves state-of-the-art results on FCDB and FLMS datasets.

Significant improvement over supervised baseline.

Abstract

Automatic image cropping models predict reframing boxes to enhance image aesthetics. Yet, the scarcity of labeled data hinders the progress of this task. To overcome this limitation, we explore the possibility of utilizing both labeled and unlabeled data together to expand the scale of training data for image cropping models. This idea can be implemented in a pseudo-labeling way: producing pseudo labels for unlabeled data by a teacher model and training a student model with these pseudo labels. However, the student may learn from teacher's mistakes. To address this issue, we propose the multi-policy viewfinder network (MPV-Net) that offers diverse refining policies to rectify the mistakes in original pseudo labels from the teacher. The most reliable policy is selected to generate trusted pseudo labels. The reliability of policies is evaluated via the robustness against box jittering. The efficacy of our method can be evaluated by the improvement compared to the supervised baseline which only uses labeled data. Notably, our MPV-Net outperforms off-the-shelf pseudo-labeling methods, yielding the most substantial improvement over the supervised baseline. Furthermore, our approach achieves state-of-the-art results on both the FCDB and FLMS datasets, signifying the superiority of our approach.