More Than A Shortcut: A Hyperbolic Approach To Early-Exit Networks

TL;DR

This paper introduces Hyperbolic Early-Exit networks (HypEE), which learn hierarchical representations in hyperbolic space to improve early prediction reliability and efficiency in resource-constrained event detection tasks.

Contribution

The paper proposes a novel hyperbolic learning framework with a hierarchical training objective and entailment loss to enforce a partial-ordering of representations, enhancing early-exit performance.

Findings

HypEE outperforms Euclidean EE baselines in multiple audio detection tasks.

The hyperbolic geometry provides a measure of uncertainty for early predictions.

HypEE improves efficiency and accuracy over standard models without early-exits.

Abstract

Deploying accurate event detection on resource-constrained devices is challenged by the trade-off between performance and computational cost. While Early-Exit (EE) networks offer a solution through adaptive computation, they often fail to enforce a coherent hierarchical structure, limiting the reliability of their early predictions. To address this, we propose Hyperbolic Early-Exit networks (HypEE), a novel framework that learns EE representations in the hyperbolic space. Our core contribution is a hierarchical training objective with a novel entailment loss, which enforces a partial-ordering constraint to ensure that deeper network layers geometrically refine the representations of shallower ones. Experiments on multiple audio event detection tasks and backbone architectures show that HypEE significantly outperforms standard Euclidean EE baselines, especially at the earliest, most computationally-critical exits. The learned geometry also provides a principled measure of uncertainty, enabling a novel triggering mechanism that makes the overall system both more efficient and more accurate than a conventional EE and standard backbone models without early-exits.