Dynamic Importance Learning using Fisher Information Matrix (FIM) for Nonlinear Dynamic Mapping

TL;DR

This paper introduces a gradient-based method using Fisher Information Matrix to dynamically assess feature relevance in nonlinear dynamic systems, enhancing interpretability and performance in black-box models.

Contribution

It presents a novel approach that integrates FIM with logistic regression for real-time relevance scoring, improving interpretability and feature interaction understanding in nonlinear models.

Findings

Effective feature relevance inference demonstrated in simulations

Superior performance over existing relevance techniques

Successful application to industrial pH neutralization process

Abstract

Understanding output variance is critical in modeling nonlinear dynamic systems, as it reflects the system's sensitivity to input variations and feature interactions. This work presents a methodology for dynamically determining relevance scores in black-box models while ensuring interpretability through an embedded decision module. This interpretable module, integrated into the first layer of the model, employs the Fisher Information Matrix (FIM) and logistic regression to compute relevance scores, interpreted as the probabilities of input neurons being active based on their contribution to the output variance. The proposed method leverages a gradient-based framework to uncover the importance of variance-driven features, capturing both individual contributions and complex feature interactions. These relevance scores are applied through element-wise transformations of the inputs, enabling the black-box model to prioritize features dynamically based on their impact on system output. This approach effectively bridges interpretability with the intricate modeling of nonlinear dynamics and time-dependent interactions. Simulation results demonstrate the method's ability to infer feature interactions while achieving superior performance in feature relevance compared to existing techniques. The practical utility of this approach is showcased through its application to an industrial pH neutralization process, where critical system dynamics are uncovered.