Semantic Clustering for Robust Fine-Grained Scene Recognition

TL;DR

This paper introduces a semantic clustering approach for robust fine-grained scene recognition in domain generalization, utilizing object occurrence patterns and semantic topics to improve transferability across unseen domains.

Contribution

It proposes a novel semantic scene descriptor and a clustering-based method for domain generalization in fine-grained scene recognition, along with a new dataset for evaluation.

Findings

Outperforms state-of-the-art methods on multiple datasets

Effective in cross-domain fine-grained scene recognition

Demonstrates robustness to object configuration variations

Abstract

In domain generalization, the knowledge learnt from one or multiple source domains is transferred to an unseen target domain. In this work, we propose a novel domain generalization approach for fine-grained scene recognition. We first propose a semantic scene descriptor that jointly captures the subtle differences between fine-grained scenes, while being robust to varying object configurations across domains. We model the occurrence patterns of objects in scenes, capturing the informativeness and discriminability of each object for each scene. We then transform such occurrences into scene probabilities for each scene image. Second, we argue that scene images belong to hidden semantic topics that can be discovered by clustering our semantic descriptors. To evaluate the proposed method, we propose a new fine-grained scene dataset in cross-domain settings. Extensive experiments on the proposed dataset and three benchmark scene datasets show the effectiveness of the proposed approach for fine-grained scene transfer, where we outperform state-of-the-art scene recognition and domain generalization methods.