Multispectral Contrastive Learning with Viewmaker Networks

TL;DR

This paper demonstrates that Viewmaker networks can generate effective views for contrastive learning across various multispectral remote sensing datasets, outperforming traditional methods without requiring domain-specific knowledge.

Contribution

It introduces the application of Viewmaker networks to multispectral remote sensing data, showing their effectiveness in improving contrastive learning performance across diverse formats.

Findings

Viewmaker outperforms cropping and reflection methods in all tested cases.

Domain-agnostic view generation is effective for scientific data.

Contrastive learning benefits from automated view generation in remote sensing.

Abstract

Contrastive learning methods have been applied to a range of domains and modalities by training models to identify similar "views" of data points. However, specialized scientific modalities pose a challenge for this paradigm, as identifying good views for each scientific instrument is complex and time-intensive. In this paper, we focus on applying contrastive learning approaches to a variety of remote sensing datasets. We show that Viewmaker networks, a recently proposed method for generating views, are promising for producing views in this setting without requiring extensive domain knowledge and trial and error. We apply Viewmaker to four multispectral imaging problems, each with a different format, finding that Viewmaker can outperform cropping- and reflection-based methods for contrastive learning in every case when evaluated on downstream classification tasks. This provides additional evidence that domain-agnostic methods can empower contrastive learning to scale to real-world scientific domains. Open source code can be found at https://github.com/jbayrooti/divmaker.