Probabilistic graphs using coupled random variables

TL;DR

This paper introduces a novel probabilistic graph model using coupled random variables based on nonextensive statistical mechanics, enabling complex correlation modeling with fewer parameters while maintaining probabilistic rigor.

Contribution

It presents a new neural node design that incorporates nonlinear coupling of input states and generalizes Bayes rule for modeling correlations among many variables.

Findings

Effective modeling of correlated variables with a single coupling parameter

Minimal performance reduction (3-4%) in classification and inference tasks

Demonstrated on UCI's MLR dataset with thousands of correlation parameters

Abstract

Neural network design has utilized flexible nonlinear processes which can mimic biological systems, but has suffered from a lack of traceability in the resulting network. Graphical probabilistic models ground network design in probabilistic reasoning, but the restrictions reduce the expressive capability of each node making network designs complex. The ability to model coupled random variables using the calculus of nonextensive statistical mechanics provides a neural node design incorporating nonlinear coupling between input states while maintaining the rigor of probabilistic reasoning. A generalization of Bayes rule using the coupled product enables a single node to model correlation between hundreds of random variables. A coupled Markov random field is designed for the inferencing and classification of UCI's MLR 'Multiple Features Data Set' such that thousands of linear correlation parameters can be replaced with a single coupling parameter with just a (3%, 4%) percent reduction in (classification, inference) performance.