Sharpening Occam's Razor
Ming Li (Univ. Waterloo), John Tromp (CWI), and Paul Vitanyi (CWI and, University of Amsterdam)

TL;DR
This paper introduces a new, Kolmogorov complexity-based formulation of Occam's razor that improves sample complexity bounds and extends the reverse theorem to broader conditions, including superpolynomial times.
Contribution
It presents a representation-independent formulation of Occam's razor theorem that enhances theoretical bounds and relaxes previous assumptions.
Findings
Better sample complexity than previous versions
Sharper reverse of Occam's razor theorem
Extended applicability to superpolynomial running times
Abstract
We provide a new representation-independent formulation of Occam's razor theorem, based on Kolmogorov complexity. This new formulation allows us to: (i) Obtain better sample complexity than both length-based and VC-based versions of Occam's razor theorem, in many applications. (ii) Achieve a sharper reverse of Occam's razor theorem than previous work. Specifically, we weaken the assumptions made in an earlier publication, and extend the reverse to superpolynomial running times.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMachine Learning and Algorithms · Computability, Logic, AI Algorithms · Algorithms and Data Compression
