yaglm: a Python package for fitting and tuning generalized linear models that supports structured, adaptive and non-convex penalties

TL;DR

yaglm is a versatile Python package that simplifies fitting and tuning a wide range of generalized linear models with various penalties, including structured, adaptive, and non-convex options, supporting multiple loss functions and constraints.

Contribution

It introduces a user-friendly, flexible framework for generalized linear models that supports advanced penalties and tuning methods, integrating with existing optimization algorithms and scikit-learn.

Findings

Supports structured, adaptive, and non-convex penalties

Offers multiple tuning parameter selection methods

Compatible with popular optimization algorithms

Abstract

The yaglm package aims to make the broader ecosystem of modern generalized linear models accessible to data analysts and researchers. This ecosystem encompasses a range of loss functions (e.g. linear, logistic, quantile regression), constraints (e.g. positive, isotonic) and penalties. Beyond the basic lasso/ridge, the package supports structured penalties such as the nuclear norm as well as the group, exclusive, fused, and generalized lasso. It also supports more accurate adaptive and non-convex (e.g. SCAD) versions of these penalties that often come with strong statistical guarantees at limited additional computational expense. yaglm comes with a variety of tuning parameter selection methods including: cross-validation, information criteria that have favorable model selection properties, and degrees of freedom estimators. While several solvers are built in (e.g. FISTA), a key design choice allows users to employ their favorite state of the art optimization algorithms. Designed to be user friendly, the package automatically creates tuning parameter grids, supports tuning with fast path algorithms along with parallelization, and follows a unified scikit-learn compatible API.