MOOT: a Repository of Many Multi-Objective Optimization Tasks

TL;DR

MOOT is a comprehensive repository of over 120 real-world multi-objective optimization tasks in software engineering, designed to facilitate more credible and diverse research by replacing toy problems with extensive case studies.

Contribution

It provides the largest curated collection of multi-objective optimization tasks in SE, enabling more realistic and varied research without introducing new algorithms.

Findings

Largest collection of SE multi-objective tasks

Enables more credible and diverse research

Supports case studies on stability and generality

Abstract

Software engineers must make decisions that trade off competing goals (faster vs. cheaper, secure vs. usable, accurate vs. interpretable, etc.). Despite MSR's proven techniques for exploring such goals, researchers still struggle with these trade-offs. Similarly, industrial practitioners deliver sub-optimal products since they lack the tools needed to explore these trade-offs. To address this, MOOT (http://tiny.cc/moot) is a repository of many SE multi-objective optimization tasks. MOOT's 120+ tasks cover software configuration, cloud tuning, project health, process modeling, hyperparameter optimization, and more. Sample scripts for reading MOOT and generating baseline results are available -- just clone the repository and run the sample rqx.sh files (from tiny.cc/moot0). To the best of our knowledge, MOOT is the largest and most varied collection of real multi-objective optimization tasks in SE. We note that MOOT's novelty is infrastructural, not algorithmic-we contribute curated data and research enablement, not new optimization methods. MOOT enables harder and more credible research. MOOT lets us replace studies on toy problems (or just half a dozen hand-picked examples) with case studies on 120+ examples. Such studies could focus on stability, sample efficiency, failure modes, cross-domain generality, or many other questions (see list in this document).