Interpolation Learning With Minimum Description Length

TL;DR

This paper analyzes the Minimum Description Length learning rule, demonstrating its tempered overfitting and providing finite sample guarantees and asymptotic behavior insights under label noise.

Contribution

It offers the first theoretical analysis of MDL learning's overfitting behavior and noise robustness with finite sample guarantees.

Findings

MDL exhibits tempered overfitting

Finite sample learning guarantees are established

Asymptotic behavior characterized under label noise

Abstract

We prove that the Minimum Description Length learning rule exhibits tempered overfitting. We obtain tempered agnostic finite sample learning guarantees and characterize the asymptotic behavior in the presence of random label noise.