Uncertainty measurement for complex event prediction in safety-critical systems

TL;DR

This paper introduces ML extunderscore CP, a method combining machine learning and sensitivity analysis to measure uncertainty in complex event prediction for safety-critical systems, using conformal prediction for reliable intervals.

Contribution

It presents a novel approach integrating ML and sensitivity analysis to quantify uncertainty in complex event prediction, applicable to safety-critical embedded systems.

Findings

Effective uncertainty measurement for complex event prediction

Successful application to classification and regression cases

Promising results demonstrating approach viability

Abstract

Complex events originate from other primitive events combined according to defined patterns and rules. Instead of using specialists' manual work to compose the model rules, we use machine learning (ML) to self-define these patterns and regulations based on incoming input data to produce the desired complex event. Complex events processing (CEP) uncertainty is critical for embedded and safety-critical systems. This paper exemplifies how we can measure uncertainty for the perception and prediction of events, encompassing embedded systems that can also be critical to safety. Then, we propose an approach (ML\_CP) incorporating ML and sensitivity analysis that verifies how the output varies according to each input parameter. Furthermore, our model also measures the uncertainty associated with the predicted complex event. Therefore, we use conformal prediction to build prediction intervals, as the model itself has uncertainties, and the data has noise. Also, we tested our approach with classification (binary and multi-level) and regression problems test cases. Finally, we present and discuss our results, which are very promising within our field of research and work.