A Unified and Fast Interpretable Model for Predictive Analytics

TL;DR

FXAM is a novel, fast, and interpretable additive model that extends GAM to better handle real-world data with categorical and temporal features, offering improved accuracy, efficiency, and interpretability for predictive analytics.

Contribution

The paper introduces FXAM, a unified and fast interpretable model that extends GAM with a new training procedure and optimization techniques, enhancing accuracy and efficiency in predictive analytics.

Findings

FXAM significantly outperforms existing GAMs in training speed.

FXAM better models categorical and temporal features.

FXAM provides superior inherent interpretability compared to post-hoc methods.

Abstract

Predictive analytics aims to build machine learning models to predict behavior patterns and use predictions to guide decision-making. Predictive analytics is human involved, thus the machine learning model is preferred to be interpretable. In literature, Generalized Additive Model (GAM) is a standard for interpretability. However, due to the one-to-many and many-to-one phenomena which appear commonly in real-world scenarios, existing GAMs have limitations to serve predictive analytics in terms of both accuracy and training efficiency. In this paper, we propose FXAM (Fast and eXplainable Additive Model), a unified and fast interpretable model for predictive analytics. FXAM extends GAM's modeling capability with a unified additive model for numerical, categorical, and temporal features. FXAM conducts a novel training procedure called Three-Stage Iteration (TSI). TSI corresponds to learning over numerical, categorical, and temporal features respectively. Each stage learns a local optimum by fixing the parameters of other stages. We design joint learning over categorical features and partial learning over temporal features to achieve high accuracy and training efficiency. We prove that TSI is guaranteed to converge to the global optimum. We further propose a set of optimization techniques to speed up FXAM's training algorithm to meet the needs of interactive analysis. Thorough evaluations conducted on diverse data sets verify that FXAM significantly outperforms existing GAMs in terms of training speed, and modeling categorical and temporal features. In terms of interpretability, we compare FXAM with the typical post-hoc approach XGBoost+SHAP on two real-world scenarios, which shows the superiority of FXAM's inherent interpretability for predictive analytics.