Contrastive View Design Strategies to Enhance Robustness to Domain Shifts in Downstream Object Detection

TL;DR

This paper investigates how contrastive view design strategies can improve the robustness of object detectors to domain shifts, proposing specific augmentation techniques and benchmarking their effectiveness across various scenarios.

Contribution

It introduces novel view augmentation strategies for contrastive pretraining that enhance robustness to appearance and context domain shifts in object detection.

Findings

Enhanced robustness to domain shifts through view augmentation strategies

Effective combination of augmentation techniques improves out-of-domain detection

Benchmark results demonstrate significant gains in robustness

Abstract

Contrastive learning has emerged as a competitive pretraining method for object detection. Despite this progress, there has been minimal investigation into the robustness of contrastively pretrained detectors when faced with domain shifts. To address this gap, we conduct an empirical study of contrastive learning and out-of-domain object detection, studying how contrastive view design affects robustness. In particular, we perform a case study of the detection-focused pretext task Instance Localization (InsLoc) and propose strategies to augment views and enhance robustness in appearance-shifted and context-shifted scenarios. Amongst these strategies, we propose changes to cropping such as altering the percentage used, adding IoU constraints, and integrating saliency based object priors. We also explore the addition of shortcut-reducing augmentations such as Poisson blending, texture flattening, and elastic deformation. We benchmark these strategies on abstract, weather, and context domain shifts and illustrate robust ways to combine them, in both pretraining on single-object and multi-object image datasets. Overall, our results and insights show how to ensure robustness through the choice of views in contrastive learning.