Automated Smell Detection and Recommendation in Natural Language Requirements

TL;DR

This paper introduces Paska, an NLP-based tool that detects quality issues in natural language requirements and provides recommendations to improve their clarity and correctness, validated through an industrial case study.

Contribution

The paper presents Paska, a novel tool combining NLP and controlled natural language to automatically detect requirement smells and suggest improvements, enhancing requirement quality assurance.

Findings

Paska achieves 89% precision and recall in smell detection.

It suggests recommendations with 96% precision and 94% recall.

Validated on 2725 requirements across 13 systems in the financial domain.

Abstract

Requirement specifications are typically written in natural language (NL) due to its usability across multiple domains and understandability by all stakeholders. However, unstructured NL is prone to quality problems (e.g., ambiguity) when writing requirements, which can result in project failures. To address this issue, we present a tool, named Paska, that takes as input any NL requirements, automatically detects quality problems as smells in the requirements, and offers recommendations to improve their quality. Our approach relies on natural language processing (NLP) techniques and a state-of-the-art controlled natural language (CNL) for requirements (Rimay), to detect smells and suggest recommendations using patterns defined in Rimay to improve requirement quality. We evaluated Paska through an industrial case study in the financial domain involving 13 systems and 2725 annotated requirements. The results show that our tool is accurate in detecting smells (89% precision and recall) and suggesting appropriate Rimay pattern recommendations (96% precision and 94% recall).