Conjecture and Inquiry: Quantifying Software Performance Requirements via Interactive Retrieval-Augmented Preference Elicitation

TL;DR

This paper introduces IRAP, an interactive method that converts natural language software performance requirements into mathematical functions, outperforming existing methods with fewer interactions and less cognitive load.

Contribution

The paper formalizes the problem of quantifying natural language performance requirements and proposes IRAP, a retrieval-augmented preference elicitation approach that improves accuracy and efficiency.

Findings

IRAP outperforms 10 state-of-the-art methods on four datasets.

IRAP achieves up to 40x improvements with only five interaction rounds.

IRAP reduces cognitive overhead in stakeholder interactions.

Abstract

Since software performance requirements are documented in natural language, quantifying them into mathematical forms is essential for software engineering. Yet, the vagueness in performance requirements and uncertainty of human cognition have caused highly uncertain ambiguity in the interpretations, rendering their automated quantification an unaddressed and challenging problem. In this paper, we formalize the problem and propose IRAP, an approach that quantifies performance requirements into mathematical functions via interactive retrieval-augmented preference elicitation. IRAP differs from the others in that it explicitly derives from problem-specific knowledge to retrieve and reason the preferences, which also guides the progressive interaction with stakeholders, while reducing the cognitive overhead. Experiment results against 10 state-of-the-art methods on four real-world datasets demonstrate the superiority of IRAP on all cases with up to 40x improvements under as few as five rounds of interactions.