Sharpening Occam's Razor

TL;DR

This paper introduces a new, Kolmogorov complexity-based formulation of Occam's razor that improves sample complexity bounds and extends the reverse theorem to broader conditions, including superpolynomial times.

Contribution

It presents a representation-independent formulation of Occam's razor theorem that enhances theoretical bounds and relaxes previous assumptions.

Findings

Better sample complexity than previous versions

Sharper reverse of Occam's razor theorem

Extended applicability to superpolynomial running times

Abstract

We provide a new representation-independent formulation of Occam's razor theorem, based on Kolmogorov complexity. This new formulation allows us to: (i) Obtain better sample complexity than both length-based and VC-based versions of Occam's razor theorem, in many applications. (ii) Achieve a sharper reverse of Occam's razor theorem than previous work. Specifically, we weaken the assumptions made in an earlier publication, and extend the reverse to superpolynomial running times.