The Group Robustness is in the Details: Revisiting Finetuning under Spurious Correlations

TL;DR

This paper investigates how finetuning affects group robustness in machine learning models, revealing complex interactions between class-balancing, model scaling, and spectral properties that influence worst-group accuracy.

Contribution

It uncovers nuanced behaviors of finetuned models on worst-group accuracy, proposing a mixture method for class-balancing and highlighting spectral imbalance as a source of disparities.

Findings

Class-balancing techniques can decrease worst-group accuracy over training epochs.

A mixture method can outperform pure class-balancing or data removal.

Scaling pretrained models benefits worst-group accuracy when combined with class-balancing.

Abstract

Modern machine learning models are prone to over-reliance on spurious correlations, which can often lead to poor performance on minority groups. In this paper, we identify surprising and nuanced behavior of finetuned models on worst-group accuracy via comprehensive experiments on four well-established benchmarks across vision and language tasks. We first show that the commonly used class-balancing techniques of mini-batch upsampling and loss upweighting can induce a decrease in worst-group accuracy (WGA) with training epochs, leading to performance no better than without class-balancing. While in some scenarios, removing data to create a class-balanced subset is more effective, we show this depends on group structure and propose a mixture method which can outperform both techniques. Next, we show that scaling pretrained models is generally beneficial for worst-group accuracy, but only in conjunction with appropriate class-balancing. Finally, we identify spectral imbalance in finetuning features as a potential source of group disparities -- minority group covariance matrices incur a larger spectral norm than majority groups once conditioned on the classes. Our results show more nuanced interactions of modern finetuned models with group robustness than was previously known. Our code is available at https://github.com/tmlabonte/revisiting-finetuning.