Rule-based Bayesian regression

TL;DR

This paper presents a rule-based Bayesian regression method that combines uncertainty quantification with expert knowledge incorporation, validated on synthetic problems and offering benefits like simplicity and improved predictions.

Contribution

It introduces a novel framework merging rule-based systems with Bayesian inference for regression, enhancing interpretability and expert knowledge integration.

Findings

Effective in synthetic linear and PDE-based problems

Reduces uncertainty and improves point predictions

Highlights computational complexity challenges

Abstract

We introduce a novel rule-based approach for handling regression problems. The new methodology carries elements from two frameworks: (i) it provides information about the uncertainty of the parameters of interest using Bayesian inference, and (ii) it allows the incorporation of expert knowledge through rule-based systems. The blending of those two different frameworks can be particularly beneficial for various domains (e.g. engineering), where, even though the significance of uncertainty quantification motivates a Bayesian approach, there is no simple way to incorporate researcher intuition into the model. We validate our models by applying them to synthetic applications: a simple linear regression problem and two more complex structures based on partial differential equations. Finally, we review the advantages of our methodology, which include the simplicity of the implementation, the uncertainty reduction due to the added information and, in some occasions, the derivation of better point predictions, and we address limitations, mainly from the computational complexity perspective, such as the difficulty in choosing an appropriate algorithm and the added computational burden.