Search Budget in Multi-Objective Refactoring Optimization: a Model-Based Empirical Study

TL;DR

This study examines how limiting search time in multi-objective software refactoring impacts solution quality, comparing different genetic algorithms across diverse case studies.

Contribution

It provides empirical insights into the effects of search budgets on solution quality and compares the performance of NSGA-II, SPEA2, and PESA2 algorithms.

Findings

Search budgets significantly reduce solution quality.

PESA2 yields the best solutions under budget constraints.

NSGA-II is the fastest algorithm among those tested.

Abstract

Software model optimization is the task of automatically generate design alternatives, usually to improve quality aspects of software that are quantifiable, like performance and reliability. In this context, multi-objective optimization techniques have been applied to help the designer find suitable trade-offs among several non-functional properties. In this process, design alternatives can be generated through automated model refactoring, and evaluated on non-functional models. Due to their complexity, this type of optimization tasks require considerable time and resources, often limiting their application in software engineering processes. In this paper, we investigate the effects of using a search budget, specifically a time limit, to the search for new solutions. We performed experiments to quantify the impact that a change in the search budget may have on the quality of solutions. Furthermore, we analyzed how different genetic algorithms (i.e., NSGA-II, SPEA2, and PESA2) perform when imposing different budgets. We experimented on two case studies of different size, complexity, and domain. We observed that imposing a search budget considerably deteriorates the quality of the generated solutions, but the specific algorithm we choose seems to play a crucial role. From our experiments, NSGA-II is the fastest algorithm, while PESA2 generates solutions with the highest quality. Differently, SPEA2 is the slowest algorithm, and produces the solutions with the lowest quality.