Variational Geometric Information Bottleneck: Learning the Shape of Understanding
Ronald Katende

TL;DR
This paper introduces a geometric information bottleneck framework that balances informativeness and geometric simplicity, leading to more efficient and interpretable representations with theoretical guarantees and practical benefits.
Contribution
It formalizes understanding as a trade-off between information and geometric complexity, deriving bounds and introducing a variational estimator for geometry-aware learning.
Findings
Generalization error scales with intrinsic dimension.
Curvature controls approximation stability.
Curvature-aware encoders perform well with limited data.
Abstract
We propose a unified information-geometric framework that formalizes understanding in learning as a trade-off between informativeness and geometric simplicity. An encoder phi is evaluated by U(phi) = I(phi(X); Y) - beta * C(phi), where C(phi) penalizes curvature and intrinsic dimensionality, enforcing smooth, low-complexity manifolds. Under mild manifold and regularity assumptions, we derive non-asymptotic bounds showing that generalization error scales with intrinsic dimension while curvature controls approximation stability, directly linking geometry to sample efficiency. To operationalize this theory, we introduce the Variational Geometric Information Bottleneck (V-GIB), a variational estimator that unifies mutual-information compression and curvature regularization through tractable geometric proxies such as the Hutchinson trace, Jacobian norms, and local PCA. Experiments across…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Graph Neural Networks · Topological and Geometric Data Analysis · Domain Adaptation and Few-Shot Learning
