A Reduction of the Elastic Net to Support Vector Machines with an Application to GPU Computing

TL;DR

This paper establishes a theoretical link between Elastic Net regression and SVMs, enabling the use of optimized SVM solvers for faster Elastic Net computation on GPUs and multi-core systems.

Contribution

It introduces a novel reduction of Elastic Net to SVM with squared hinge loss, allowing leveraging existing SVM acceleration techniques for Elastic Net.

Findings

Achieves significant speedup over glmnet in experiments

Provides a simple MATLAB wrapper for GPU-accelerated Elastic Net

Demonstrates the reduction's effectiveness on real-world datasets

Abstract

The past years have witnessed many dedicated open-source projects that built and maintain implementations of Support Vector Machines (SVM), parallelized for GPU, multi-core CPUs and distributed systems. Up to this point, no comparable effort has been made to parallelize the Elastic Net, despite its popularity in many high impact applications, including genetics, neuroscience and systems biology. The first contribution in this paper is of theoretical nature. We establish a tight link between two seemingly different algorithms and prove that Elastic Net regression can be reduced to SVM with squared hinge loss classification. Our second contribution is to derive a practical algorithm based on this reduction. The reduction enables us to utilize prior efforts in speeding up and parallelizing SVMs to obtain a highly optimized and parallel solver for the Elastic Net and Lasso. With a simple wrapper, consisting of only 11 lines of MATLAB code, we obtain an Elastic Net implementation that naturally utilizes GPU and multi-core CPUs. We demonstrate on twelve real world data sets, that our algorithm yields identical results as the popular (and highly optimized) glmnet implementation but is one or several orders of magnitude faster.