HIPAD - A Hybrid Interior-Point Alternating Direction algorithm for knowledge-based SVM and feature selection

TL;DR

This paper introduces HIPAD, a hybrid optimization algorithm combining alternating direction and interior-point methods for knowledge-based SVMs, improving feature selection and classification in high-dimensional, scarce data scenarios.

Contribution

The paper presents a novel hybrid algorithm that integrates knowledge-based elastic-net SVM with classical SVM, enhancing optimization accuracy and flexibility for feature selection with prior knowledge.

Findings

Effective in high-dimensional, scarce data settings

Outperforms existing methods on synthetic and real data

Addresses feature selection and knowledge incorporation

Abstract

We consider classification tasks in the regime of scarce labeled training data in high dimensional feature space, where specific expert knowledge is also available. We propose a new hybrid optimization algorithm that solves the elastic-net support vector machine (SVM) through an alternating direction method of multipliers in the first phase, followed by an interior-point method for the classical SVM in the second phase. Both SVM formulations are adapted to knowledge incorporation. Our proposed algorithm addresses the challenges of automatic feature selection, high optimization accuracy, and algorithmic flexibility for taking advantage of prior knowledge. We demonstrate the effectiveness and efficiency of our algorithm and compare it with existing methods on a collection of synthetic and real-world data.