OlmoEarth v1.1: A more efficient family of OlmoEarth models

TL;DR

OlmoEarth v1.1 introduces efficiency improvements that significantly reduce training and inference costs while preserving performance, with code openly available.

Contribution

The paper presents a set of optimizations to the OlmoEarth models that cut computational costs during training and inference without sacrificing accuracy.

Findings

1. 1.7x reduction in GPU hours for training.

2. 2.9x reduction in MACs during inference.

3. Maintains overall model performance.

Abstract

We present a set of improvements to the OlmoEarth family. These improvements allow us to cut compute costs during training ($1.7 \times$ reduction in GPU hours required to train our Base models) and inference ($2.9\times$ reductions in MACs on Sentinel-2 tasks), while maintaining the models' overall performance. All training code is available at github.com/allenai/olmoearth_pretrain.