Dirichlet Logistic Gaussian Processes for Evaluation of Black-Box Stochastic Systems under Complex Requirements

TL;DR

This paper introduces a Bayesian approach using Dirichlet logistic Gaussian processes to evaluate complex performance distributions of black-box cyber-physical systems with limited data, improving uncertainty estimation.

Contribution

It proposes a novel semiparametric Bayesian model combining Dirichlet random fields and logistic Gaussian processes for distributional performance evaluation under small data conditions.

Findings

The model converges to the true distribution with increasing data.

Empirical simulations demonstrate the method's effectiveness.

Provides a conservative and reasonable estimation of performance distributions.

Abstract

The requirement-driven performance evaluation of a black-box cyber-physical system (CPS) that utilizes machine learning methods has proven to be an effective way to assess the quality of the CPS. However, the distributional evaluation of the performance has been poorly considered. Although many uncertainty estimation methods have been advocated, they have not successfully estimated highly complex performance distributions under small data. In this paper, we propose a method to distributionally evaluate the performance under complex requirements using small input-trajectory data. To handle the unknown complex probability distributions under small data, we discretize the corresponding performance measure, yielding a discrete random process over an input region. Then, we propose a semiparametric Bayesian model of the discrete process based on a Dirichlet random field whose parameter function is represented by multiple logistic Gaussian processes (LGPs). The Dirichlet posterior parameter function is estimated through the LGP posteriors in a reasonable and conservative fashion. We show that the proposed Bayesian model converges to the true discrete random process as the number of data becomes large enough. We also empirically demonstrate the effectiveness of the proposed method by simulation.