Densely Semantic Enhancement for Domain Adaptive Region-free Detectors

TL;DR

This paper introduces a novel adversarial module called DSEM that enhances domain adaptation for region-free object detectors by emphasizing important regions and aligning multi-scale semantic features, leading to improved cross-domain detection performance.

Contribution

The work proposes a pluggable DSEM module that enables densely semantic feature matching for region-free detectors, addressing the lack of instance-level feature alignment in unsupervised domain adaptation.

Findings

Outperforms existing domain adaptive detectors on multiple benchmarks.

Effectively emphasizes important image regions via foreground enhancement masks.

Achieves better domain alignment through multi-scale semantic and contextual feature encoding.

Abstract

Unsupervised domain adaptive object detection aims to adapt a well-trained detector from its original source domain with rich labeled data to a new target domain with unlabeled data. Previous works focus on improving the domain adaptability of region-based detectors, e.g., Faster-RCNN, through matching cross-domain instance-level features that are explicitly extracted from a region proposal network (RPN). However, this is unsuitable for region-free detectors such as single shot detector (SSD), which perform a dense prediction from all possible locations in an image and do not have the RPN to encode such instance-level features. As a result, they fail to align important image regions and crucial instance-level features between the domains of region-free detectors. In this work, we propose an adversarial module to strengthen the cross-domain matching of instance-level features for region-free detectors. Firstly, to emphasize the important regions of image, the DSEM learns to predict a transferable foreground enhancement mask that can be utilized to suppress the background disturbance in an image. Secondly, considering that region-free detectors recognize objects of different scales using multi-scale feature maps, the DSEM encodes both multi-level semantic representations and multi-instance spatial-contextual relationships across different domains. Finally, the DSEM is pluggable into different region-free detectors, ultimately achieving the densely semantic feature matching via adversarial learning. Extensive experiments have been conducted on PASCAL VOC, Clipart, Comic, Watercolor, and FoggyCityscape benchmarks, and their results well demonstrate that the proposed approach not only improves the domain adaptability of region-free detectors but also outperforms existing domain adaptive region-based detectors under various domain shift settings.