Learning from Uncertain Data: From Possible Worlds to Possible Models

TL;DR

This paper presents a novel method for efficiently learning linear models from uncertain data by using abstract interpretation and zonotopes to handle all possible data variations simultaneously, ensuring convergence and providing uncertainty bounds.

Contribution

It introduces a new approach combining abstract interpretation and zonotopes for symbolic gradient descent on uncertain datasets, enabling sound over-approximations of models and predictions.

Findings

Method effectively handles data uncertainty and predictive multiplicity.

Provides theoretical guarantees of convergence and fixed point solutions.

Demonstrates practical effectiveness through empirical analysis.

Abstract

We introduce an efficient method for learning linear models from uncertain data, where uncertainty is represented as a set of possible variations in the data, leading to predictive multiplicity. Our approach leverages abstract interpretation and zonotopes, a type of convex polytope, to compactly represent these dataset variations, enabling the symbolic execution of gradient descent on all possible worlds simultaneously. We develop techniques to ensure that this process converges to a fixed point and derive closed-form solutions for this fixed point. Our method provides sound over-approximations of all possible optimal models and viable prediction ranges. We demonstrate the effectiveness of our approach through theoretical and empirical analysis, highlighting its potential to reason about model and prediction uncertainty due to data quality issues in training data.