Stochastic interior-point methods for smooth conic optimization with applications

TL;DR

This paper introduces a stochastic interior-point method framework for general conic optimization problems, addressing a gap in scalable algorithms for large datasets in machine learning.

Contribution

It develops a new stochastic interior-point method with four variants and establishes their iteration complexity, matching the best-known results in stochastic unconstrained optimization.

Findings

Effective in robust linear regression

Improves multi-task relationship learning

Efficient for clustering data streams

Abstract

Conic optimization plays a crucial role in many machine learning (ML) problems. However, practical algorithms for conic constrained ML problems with large datasets are often limited to specific use cases, as stochastic algorithms for general conic optimization remain underdeveloped. To fill this gap, we introduce a stochastic interior-point method (SIPM) framework for general conic optimization, along with four novel SIPM variants leveraging distinct stochastic gradient estimators. Under mild assumptions, we establish the iteration complexity of our proposed SIPMs, which, up to a polylogarithmic factor, match the best-known {results} in stochastic unconstrained optimization. Finally, our numerical experiments on robust linear regression, multi-task relationship learning, and clustering data streams demonstrate the effectiveness and efficiency of our approach.