Linear Support Vector Regression with Linear Constraints

TL;DR

This paper introduces a linear constrained version of Support Vector Regression, providing a method to incorporate prior knowledge like monotonicity or probability constraints, with a generalized SMO algorithm and demonstrated practical benefits.

Contribution

It presents a novel linear constrained SVR formulation, a generalized SMO algorithm with convergence proof, and empirical validation on diverse datasets.

Findings

Effective incorporation of prior knowledge into SVR

Convergence of the proposed optimization algorithm

Improved regression performance on real datasets

Abstract

This paper studies the addition of linear constraints to the Support Vector Regression (SVR) when the kernel is linear. Adding those constraints into the problem allows to add prior knowledge on the estimator obtained, such as finding probability vector or monotone data. We propose a generalization of the Sequential Minimal Optimization (SMO) algorithm for solving the optimization problem with linear constraints and prove its convergence. Then, practical performances of this estimator are shown on simulated and real datasets with different settings: non negative regression, regression onto the simplex for biomedical data and isotonic regression for weather forecast.