Can Large Language Models Improve SE Active Learning via Warm-Starts?

TL;DR

This paper investigates whether Large Language Models can enhance active learning in software engineering by providing effective warm-starts, showing significant improvements in low- and medium-dimensional tasks but limited benefits in high-dimensional scenarios.

Contribution

It introduces the use of LLMs for warm-starts in SE active learning and compares their effectiveness against Bayesian methods across various task dimensions.

Findings

LLMs significantly improve performance in low- and medium-dimensional tasks.

Bayesian methods outperform LLMs in high-dimensional problems.

LLMs' effectiveness diminishes as task dimensionality increases.

Abstract

When SE data is scarce, "active learners" use models learned from tiny samples of the data to find the next most informative example to label. In this way, effective models can be generated using very little data. For multi-objective software engineering (SE) tasks, active learning can benefit from an effective set of initial guesses (also known as "warm starts"). This paper explores the use of Large Language Models (LLMs) for creating warm-starts. Those results are compared against Gaussian Process Models and Tree of Parzen Estimators. For 49 SE tasks, LLM-generated warm starts significantly improved the performance of low- and medium-dimensional tasks. However, LLM effectiveness diminishes in high-dimensional problems, where Bayesian methods like Gaussian Process Models perform best.