Self-Supervision based Task-Specific Image Collection Summarization

TL;DR

This paper introduces a self-supervised, task-specific image summarization method using a Wasserstein GAN and semantic clustering, enabling efficient dataset reduction without retraining for different summary sizes.

Contribution

The work presents a novel self-supervised approach combining Wasserstein GANs and clustering for flexible, task-specific image summarization that does not require retraining for different summary lengths.

Findings

Effective in generating representative image summaries

No retraining needed for different summary sizes

Demonstrates robustness and discriminative power

Abstract

Successful applications of deep learning (DL) requires large amount of annotated data. This often restricts the benefits of employing DL to businesses and individuals with large budgets for data-collection and computation. Summarization offers a possible solution by creating much smaller representative datasets that can allow real-time deep learning and analysis of big data and thus democratize use of DL. In the proposed work, our aim is to explore a novel approach to task-specific image corpus summarization using semantic information and self-supervision. Our method uses a classification-based Wasserstein generative adversarial network (CLSWGAN) as a feature generating network. The model also leverages rotational invariance as self-supervision and classification on another task. All these objectives are added on a features from resnet34 to make it discriminative and robust. The model then generates a summary at inference time by using K-means clustering in the semantic embedding space. Thus, another main advantage of this model is that it does not need to be retrained each time to obtain summaries of different lengths which is an issue with current end-to-end models. We also test our model efficacy by means of rigorous experiments both qualitatively and quantitatively.