Gradient Projection Newton Algorithm for Sparse Collaborative Learning Using Synthetic and Real Datasets of Applications

TL;DR

This paper introduces a novel sparse collaborative learning model with a gradient projection Newton algorithm, capable of handling classification and regression tasks involving two datasets, demonstrating high efficiency through experiments on synthetic and real data.

Contribution

The paper proposes a new double-sparsity constrained optimization model and a convergent Newton-based algorithm for collaborative learning from multiple datasets.

Findings

Algorithm converges globally with quadratic rate

Effective in classification and regression tasks

Validated on synthetic and real datasets

Abstract

Exploring the relationship among multiple sets of data from one same group enables practitioners to make better decisions in medical science and engineering. In this paper, we propose a sparse collaborative learning (SCL) model, an optimization with double-sparsity constraints, to process the problem with two sets of data and a shared response variable. It is capable of dealing with the classification problems or the regression problems dependent on the discreteness of the response variable as well as exploring the relationship between two datasets simultaneously. To solve SCL, we first present some necessary and sufficient optimality conditions and then design a gradient projection Newton algorithm which has proven to converge to a unique locally optimal solution globally with at least a quadratic convergence rate. Finally, the reported numerical experiments illustrate the efficiency of the proposed method.