Plug-in martingales for testing exchangeability on-line
Valentina Fedorova, Alex Gammerman, Ilia Nouretdinov, and Vladimir, Vovk

TL;DR
This paper introduces a new method for online testing of data exchangeability using martingales, which is more flexible and effective on complex datasets than existing techniques.
Contribution
The paper extends existing exchangeability martingale techniques and demonstrates their effectiveness in online testing scenarios, especially on challenging datasets.
Findings
The new method performs well on USPS dataset.
It outperforms previous techniques on Statlog Satellite data.
Existing methods are sufficient for simple datasets but less effective on complex data.
Abstract
A standard assumption in machine learning is the exchangeability of data, which is equivalent to assuming that the examples are generated from the same probability distribution independently. This paper is devoted to testing the assumption of exchangeability on-line: the examples arrive one by one, and after receiving each example we would like to have a valid measure of the degree to which the assumption of exchangeability has been falsified. Such measures are provided by exchangeability martingales. We extend known techniques for constructing exchangeability martingales and show that our new method is competitive with the martingales introduced before. Finally we investigate the performance of our testing method on two benchmark datasets, USPS and Statlog Satellite data; for the former, the known techniques give satisfactory results, but for the latter our new more flexible method…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsData Stream Mining Techniques · Time Series Analysis and Forecasting · Software System Performance and Reliability
