Statistical Learning and Inverse Problems: A Stochastic Gradient Approach

TL;DR

This paper explores the use of stochastic gradient descent algorithms for solving statistical inverse problems, providing theoretical guarantees and practical modifications to enhance performance, with applications in functional linear regression.

Contribution

It introduces a stochastic gradient approach for linear statistical inverse problems, offering theoretical bounds and a novel smoothing modification for improved empirical results.

Findings

Finite sample bounds for excess risk established

Modified SGD with smoothing improves empirical performance

Application demonstrated in functional linear regression with real and synthetic data

Abstract

Inverse problems are paramount in Science and Engineering. In this paper, we consider the setup of Statistical Inverse Problem (SIP) and demonstrate how Stochastic Gradient Descent (SGD) algorithms can be used in the linear SIP setting. We provide consistency and finite sample bounds for the excess risk. We also propose a modification for the SGD algorithm where we leverage machine learning methods to smooth the stochastic gradients and improve empirical performance. We exemplify the algorithm in a setting of great interest nowadays: the Functional Linear Regression model. In this case we consider a synthetic data example and examples with a real data classification problem.