A Prompt-based Few-shot Learning Approach to Software Conflict Detection

TL;DR

This paper introduces a prompt-based few-shot learning method for software requirement conflict detection, achieving comparable performance to large supervised models with minimal labeled data, and handling both functional and non-functional requirements.

Contribution

The study presents a novel prompt-based few-shot learning approach for conflict detection in SRS documents, eliminating the need for extensive labeled data and requirement filtering.

Findings

Prompting with 32 examples matches large supervised models' performance.

The approach handles both functional and non-functional requirements.

Minimal labeled data suffices for effective conflict detection.

Abstract

A software requirement specification (SRS) document is an essential part of the software development life cycle which outlines the requirements that a software program in development must satisfy. This document is often specified by a diverse group of stakeholders and is subject to continual change, making the process of maintaining the document and detecting conflicts between requirements an essential task in software development. Notably, projects that do not address conflicts in the SRS document early on face considerable problems later in the development life cycle. These problems incur substantial costs in terms of time and money, and these costs often become insurmountable barriers that ultimately result in the termination of a software project altogether. As a result, early detection of SRS conflicts is critical to project sustainability. The conflict detection task is approached in numerous ways, many of which require a significant amount of manual intervention from developers, or require access to a large amount of labeled, task-specific training data. In this work, we propose using a prompt-based learning approach to perform few-shot learning for conflict detection. We compare our results to supervised learning approaches that use pretrained language models, such as BERT and its variants. Our results show that prompting with just 32 labeled examples can achieve a similar level of performance in many key metrics to that of supervised learning on training sets that are magnitudes larger in size. In contrast to many other conflict detection approaches, we make no assumptions about the type of underlying requirements, allowing us to analyze pairings of both functional and non-functional requirements. This allows us to omit the potentially expensive task of filtering out non-functional requirements from our dataset.