An Aggregate and Iterative Disaggregate Algorithm with Proven Optimality in Machine Learning

TL;DR

This paper introduces an iterative clustering-based algorithm that aggregates and disaggregates data to efficiently solve certain machine learning optimization problems, with proven optimality and convergence guarantees.

Contribution

It presents a novel aggregation-disaggregation algorithm with proven optimality and convergence for specific machine learning problems, enhancing solution efficiency.

Findings

Algorithm achieves optimal solutions in tested cases.

Proven convergence and bounds on optimality gap.

Effective for problems like SVMs and regression.

Abstract

We propose a clustering-based iterative algorithm to solve certain optimization problems in machine learning, where we start the algorithm by aggregating the original data, solving the problem on aggregated data, and then in subsequent steps gradually disaggregate the aggregated data. We apply the algorithm to common machine learning problems such as the least absolute deviation regression problem, support vector machines, and semi-supervised support vector machines. We derive model-specific data aggregation and disaggregation procedures. We also show optimality, convergence, and the optimality gap of the approximated solution in each iteration. A computational study is provided.