Light over Heavy: Automated Performance Requirements Quantification with Linguistic Inducement

TL;DR

This paper introduces LQPR, an efficient automatic method for quantifying performance requirements by leveraging linguistic patterns, outperforming existing learning-based approaches in accuracy and cost-effectiveness.

Contribution

LQPR presents a novel theoretical framework and lightweight matching mechanism that effectively quantifies performance requirements, challenging the reliance on large language models.

Findings

LQPR outperforms nine state-of-the-art methods in 75% of cases.

LQPR achieves comparable accuracy with two orders of magnitude less cost.

Specialized linguistic methods can surpass general LLM approaches in requirement quantification.

Abstract

Elicited performance requirements need to be quantified for compliance in different engineering tasks, e.g., configuration tuning and performance testing. Much existing work has relied on manual quantification, which is expensive and error-prone due to the imprecision. In this paper, we present LQPR, a highly efficient automatic approach for performance requirements quantification.LQPR relies on a new theoretical framework that converts quantification as a classification problem. Despite the prevalent applications of Large Language Models (LLMs) for requirement analytics, LQPR takes a different perspective to address the classification: we observed that performance requirements can exhibit strong patterns and are often short/concise, therefore we design a lightweight linguistically induced matching mechanism. We compare LQPR against nine state-of-the-art learning-based approaches over diverse datasets, demonstrating that it is ranked as the sole best for 75% or more cases with two orders less cost. Our work proves that, at least for performance requirement quantification, specialized methods can be more suitable than the general LLM-driven approaches.