Online optimization in dynamic environments: a regret analysis for sparse problems

TL;DR

This paper develops an online algorithm for sparse optimization problems, specifically Elastic-net, in dynamic environments, and proves its effectiveness through regret analysis and practical applications.

Contribution

It introduces a novel online Elastic-net algorithm with theoretical regret guarantees for time-varying sparse problems.

Findings

The algorithm achieves low dynamic regret in theoretical analysis.

Numerical results demonstrate practical efficiency in identifying time-varying models.

Application to autoregressive models with unknown parameters shows versatility.

Abstract

Time-varying systems are a challenge in many scientific and engineering areas. Usually, estimation of time-varying parameters or signals must be performed online, which calls for the development of responsive online algorithms. In this paper, we consider this problem in the context of the sparse optimization; specifically, we consider the Elastic-net model, which promotes parsimonious solutions. Following the rationale in \cite{mok16}, we propose an online algorithm and we theoretically prove that it is successful in terms of dynamic regret. We then show an application to the problem of recursive identification of time-varying autoregressive models, in the case when the number of parameters to be estimated is unknown. Numerical results show the practical efficiency of the proposed method.