Comparative Model Fidelity Evaluation to Support Design Decisions for Complex, Novel Systems of Systems

TL;DR

This paper introduces a method to evaluate the trustworthiness of simulation models in complex systems design, especially when real-world validation data is unavailable, by focusing on model fidelity to physical phenomena.

Contribution

It presents a novel approach to assess model fidelity based on physical adherence, aiding designers in selecting appropriate models without requiring ground truth data.

Findings

Method effectively distinguishes models with higher physical fidelity.

Supports decision-making in early-stage system design.

Enhances confidence in simulation-based design choices.

Abstract

Systems design processes are increasingly reliant on simulation models to inform design decisions. A pervasive issue within the systems engineering community is trusting in the models used to make decisions about complex systems. This work presents a method of evaluating the trustworthiness of a model to provide utility to a designer making a decision within a design process. Trusting the results of a model is especially important in design processes where the system is complex, novel, or displays emergent phenomena. Additionally, systems that are in the pre-prototype stages of development often do not have sources of ground truth for validating the models. Developing methods of model validation and trust that do not require real-world data is a key challenge facing systems engineers. Model fidelity in this work refers to the adherence of a model to real-world physics and is closely tied to model trust and model validity. Trust and validity directly support a designer's ability to make decisions using physics-based models. The physics that are captured in a model and the complexity of the mathematical representation of the physics contribute to a model's fidelity, and this work leverages the included physical phenomena to develop a means of selecting the most appropriate for a given design decision.