In Search for a Generalizable Method for Source Free Domain Adaptation

TL;DR

This paper evaluates existing source-free domain adaptation methods on bioacoustics data, finds inconsistent performance, and introduces a simple new method that improves robustness across diverse data modalities.

Contribution

The paper demonstrates the limitations of current SFDA techniques on bioacoustic data and proposes a new simple method that outperforms existing approaches across multiple datasets.

Findings

Existing SFDA methods perform variably on bioacoustic shifts.

Some methods perform worse than no adaptation.

The new method outperforms existing methods on bioacoustic and vision datasets.

Abstract

Source-free domain adaptation (SFDA) is compelling because it allows adapting an off-the-shelf model to a new domain using only unlabelled data. In this work, we apply existing SFDA techniques to a challenging set of naturally-occurring distribution shifts in bioacoustics, which are very different from the ones commonly studied in computer vision. We find existing methods perform differently relative to each other than observed in vision benchmarks, and sometimes perform worse than no adaptation at all. We propose a new simple method which outperforms the existing methods on our new shifts while exhibiting strong performance on a range of vision datasets. Our findings suggest that existing SFDA methods are not as generalizable as previously thought and that considering diverse modalities can be a useful avenue for designing more robust models.