Maximal Margin Distribution Support Vector Regression with coupled Constraints-based Convex Optimization

TL;DR

This paper introduces a convex optimization approach for support vector regression that maximizes the margin distribution, leading to improved prediction accuracy and smoother regression curves.

Contribution

It proposes a novel maximal margin distribution SVR model with coupled constraints, transforming a non-convex problem into a convex one for better efficiency and performance.

Findings

MMD-SVR outperforms classic SVR in prediction accuracy

The new model produces smoother regression curves

The approach enhances training feasibility and efficiency

Abstract

Support vector regression (SVR) is one of the most popular machine learning algorithms aiming to generate the optimal regression curve through maximizing the minimal margin of selected training samples, i.e., support vectors. Recent researchers reveal that maximizing the margin distribution of whole training dataset rather than the minimal margin of a few support vectors, is prone to achieve better generalization performance. However, the margin distribution support vector regression machines suffer difficulties resulted from solving a non-convex quadratic optimization, compared to the margin distribution strategy for support vector classification, This paper firstly proposes a maximal margin distribution model for SVR(MMD-SVR), then implementing coupled constrain factor to convert the non-convex quadratic optimization to a convex problem with linear constrains, which enhance the training feasibility and efficiency for SVR to derived from maximizing the margin distribution. The theoretical and empirical analysis illustrates the superiority of MMD-SVR. In addition, numerical experiments show that MMD-SVR could significantly improve the accuracy of prediction and generate more smooth regression curve with better generalization compared with the classic SVR.