A Multi-level Methodology for Behavioral Comparison of Software-Intensive Systems

TL;DR

This paper introduces a fully automated, multi-level methodology for comparing behaviors of software-intensive systems using state machine models, helping developers understand change impacts and prevent regressions.

Contribution

It presents a novel, step-by-step comparison approach that combines existing methods into a comprehensive, automated framework for behavioral analysis of complex software systems.

Findings

Effective in identifying behavioral differences in case studies

Potential to prevent regressions in system behavior

Validated through empirical field studies at ASML

Abstract

Software-intensive systems constantly evolve. To prevent software changes from unintentionally introducing costly system defects, it is important to understand their impact to reduce risk. However, it is in practice nearly impossible to foresee the full impact of software changes when dealing with huge industrial systems with many configurations and usage scenarios. To assist developers with change impact analysis we introduce a novel multi-level methodology for behavioral comparison of software-intensive systems. Our fully automated methodology is based on comparing state machine models of software behavior. We combine existing complementary comparison methods into a novel approach, guiding users step by step though relevant differences by gradually zooming into more and more detail. We empirically evaluate our work through a qualitative exploratory field study, showing its practical value using multiple case studies at ASML, a leading company in developing lithography systems. Our method shows great potential for preventing regressions in system behavior for software changes.