Adaptive Fine-tuning for Multiclass Classification over Software Requirement Data

TL;DR

This paper introduces a three-stage domain-adaptive fine-tuning method for multi-class classification of software requirement texts, enhancing model robustness and accuracy amid distribution shifts in real-world data.

Contribution

It proposes a novel three-stage adaptive fine-tuning approach specifically designed for software requirement classification tasks, outperforming existing baselines.

Findings

Adaptive fine-tuning improves classification performance

Models become more robust to distribution shifts

Significant accuracy gains over baseline methods

Abstract

The analysis of software requirement specifications (SRS) using Natural Language Processing (NLP) methods has been an important study area in the software engineering field in recent years. Especially thanks to the advances brought by deep learning and transfer learning approaches in NLP, SRS data can be utilized for various learning tasks more easily. In this study, we employ a three-stage domain-adaptive fine-tuning approach for three prediction tasks regarding software requirements, which improve the model robustness on a real distribution shift. The multi-class classification tasks involve predicting the type, priority and severity of the requirement texts specified by the users. We compare our results with strong classification baselines such as word embedding pooling and Sentence BERT, and show that the adaptive fine-tuning leads to performance improvements across the tasks. We find that an adaptively fine-tuned model can be specialized to particular data distribution, which is able to generate accurate results and learns from abundantly available textual data in software engineering task management systems.