Data-driven configuration tuning of glmnet for balancing accuracy and computational efficiency

TL;DR

This paper introduces a data-driven framework using neural networks to optimize glmnet configuration, balancing solution accuracy and computational efficiency, and enabling automatic selection based on user constraints.

Contribution

It presents a systematic, neural network-based approach to tune glmnet configurations for optimal tradeoffs between accuracy and speed, implemented in an R package.

Findings

Neural networks accurately predict glmnet accuracy and computation time.

The framework effectively generates Pareto fronts for configuration tradeoffs.

Automatic configuration selection improves solution quality within time constraints.

Abstract

The glmnet package in R is widely used for lasso estimation because of its computational efficiency. Despite its popularity, glmnet occasionally yields solutions that deviate substantially from the true ones because of the inappropriate default configuration of the algorithm. The accuracy of the obtained solutions can be improved by appropriately tuning the configuration. However, such improvements typically increase computational time, resulting in a tradeoff between accuracy and computational efficiency. Therefore, a systematic approach is required to determine the appropriate configuration. To address this need, we propose a unified data-driven framework specifically designed to optimize the configuration by balancing solution path accuracy and computational cost. Specifically, we generate a large-scale training dataset by measuring the accuracy and computation time of glmnet. Using this dataset, we construct neural networks to predict accuracy and computation time from data characteristics and configuration. For a new dataset, the proposed framework uses the trained networks to explore the configuration space and derive a Pareto front that represents the tradeoff between accuracy and computational cost. This front enables automatic selection of the configuration that maximizes accuracy under a user-specified time constraint. The proposed method is implemented in the R package glmnetconf, available at https://github.com/Shuhei-Muroya/glmnetconf.git.