Genetic Generalized Additive Models

TL;DR

This paper introduces a genetic algorithm-based method to automatically optimize Generalized Additive Models, balancing accuracy and interpretability, resulting in simpler, smoother, and more confident models.

Contribution

It presents a novel automated framework using NSGA-II to optimize GAMs for both accuracy and complexity, improving model interpretability and performance.

Findings

NSGA-II discovers GAMs with better accuracy or similar performance at lower complexity.

Optimized GAMs are simpler, smoother, and have narrower confidence intervals.

The approach enhances interpretability without sacrificing predictive power.

Abstract

Generalized Additive Models (GAMs) balance predictive accuracy and interpretability, but manually configuring their structure is challenging. We propose using the multi-objective genetic algorithm NSGA-II to automatically optimize GAMs, jointly minimizing prediction error (RMSE) and a Complexity Penalty that captures sparsity, smoothness, and uncertainty. Experiments on the California Housing dataset show that NSGA-II discovers GAMs that outperform baseline LinearGAMs in accuracy or match performance with substantially lower complexity. The resulting models are simpler, smoother, and exhibit narrower confidence intervals, enhancing interpretability. This framework provides a general approach for automated optimization of transparent, high-performing models. The code can be found at https://github.com/KaaustaaubShankar/GeneticAdditiveModels.