Debiasing Stochastic Gradient Descent to handle missing values

TL;DR

This paper introduces a new averaged stochastic gradient algorithm that effectively handles missing data in linear models, achieving optimal convergence rates without requiring data distribution assumptions, applicable to large-scale and real-world datasets.

Contribution

The paper presents a novel stochastic gradient method that manages missing values without distribution modeling, maintaining optimal convergence rates in both streaming and finite-sample scenarios.

Findings

Achieves convergence rate of O(1/n) despite missing data

Effective on both synthetic and real medical datasets

Does not require prior data distribution assumptions

Abstract

Stochastic gradient algorithm is a key ingredient of many machine learning methods, particularly appropriate for large-scale learning.However, a major caveat of large data is their incompleteness.We propose an averaged stochastic gradient algorithm handling missing values in linear models. This approach has the merit to be free from the need of any data distribution modeling and to account for heterogeneous missing proportion.In both streaming and finite-sample settings, we prove that this algorithm achieves convergence rate of $\mathcal{O}(\frac{1}{n})$ at the iteration $n$, the same as without missing values. We show the convergence behavior and the relevance of the algorithm not only on synthetic data but also on real data sets, including those collected from medical register.