Foundation Model or Finetune? Evaluation of few-shot semantic segmentation for river pollution

TL;DR

This paper evaluates the effectiveness of foundation models versus finetuned models for few-shot semantic segmentation of river pollution, finding finetuned models generally perform better even with limited data.

Contribution

It provides a comparative analysis of foundation models and finetuned models on a new river pollution dataset, highlighting the advantages of finetuning in this context.

Findings

Finetuned models outperform foundation models in semantic segmentation.

Performance gap persists even with scarce data.

The study introduces a new dataset and code for future research.

Abstract

Foundation models (FMs) are a popular topic of research in AI. Their ability to generalize to new tasks and datasets without retraining or needing an abundance of data makes them an appealing candidate for applications on specialist datasets. In this work, we compare the performance of FMs to finetuned pre-trained supervised models in the task of semantic segmentation on an entirely new dataset. We see that finetuned models consistently outperform the FMs tested, even in cases were data is scarce. We release the code and dataset for this work on GitHub.