Relational Self-supervised Distillation with Compact Descriptors for Image Copy Detection

TL;DR

This paper introduces a lightweight, self-supervised distillation method that produces compact image descriptors for copy detection, achieving high accuracy with smaller models and descriptors, suitable for practical applications.

Contribution

It proposes relational self-supervised distillation to transfer knowledge from large to small networks, enabling effective, compact image descriptors for copy detection.

Findings

Achieved up to 5.9% improvement in precision with smaller descriptors

Enabled training of lightweight networks with competitive performance

Applied contrastive learning to prevent dimensional collapse

Abstract

Image copy detection is the task of detecting edited copies of any image within a reference database. While previous approaches have shown remarkable progress, the large size of their networks and descriptors remains a disadvantage, complicating their practical application. In this paper, we propose a novel method that achieves competitive performance by using a lightweight network and compact descriptors. By utilizing relational self-supervised distillation to transfer knowledge from a large network to a small network, we enable the training of lightweight networks with smaller descriptor sizes. We introduce relational self-supervised distillation for flexible representation in a smaller feature space and apply contrastive learning with a hard negative loss to prevent dimensional collapse. For the DISC2021 benchmark, ResNet-50 and EfficientNet-B0 are used as the teacher and student models, respectively, with micro average precision improving by 5.0\%/4.9\%/5.9\% for 64/128/256 descriptor sizes compared to the baseline method. The code is available at \href{https://github.com/juntae9926/RDCD}{https://github.com/juntae9926/RDCD}.