Model Matching Challenge: Benchmarks for Ecore and BPMN Diagrams
Pit Pietsch, Klaus M\"uller, Bernhard Rumpe

TL;DR
This paper introduces benchmarks for model comparison tools in Model Driven Engineering, focusing on Ecore and BPMN diagrams, by analyzing failure cases and proposing evaluation standards.
Contribution
It presents five edit operations causing comparison failures and offers benchmarks to assess the quality of model matching algorithms.
Findings
Identification of five key edit operations affecting comparison quality
Analysis of failure reasons in current comparison algorithms
Proposed benchmarks for evaluating model matching tools
Abstract
In the last couple of years, Model Driven Engineering (MDE) gained a prominent role in the context of software engineering. In the MDE paradigm, models are considered first level artifacts which are iteratively developed by teams of programmers over a period of time. Because of this, dedicated tools for versioning and management of models are needed. A central functionality within this group of tools is model comparison and differencing. In two disjunct research projects, we identified a group of general matching problems where state-of-the-art comparison algorithms delivered low quality results. In this article, we will present five edit operations which are the cause for these low quality results. The reasons why the algorithms fail, as well as possible solutions, are also discussed. These examples can be used as benchmarks by model developers to assess the quality and applicability…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsModel-Driven Software Engineering Techniques · Advanced Software Engineering Methodologies · Service-Oriented Architecture and Web Services
