Low-Regret and Low-Complexity Learning for Hierarchical Inference

TL;DR

This paper introduces a novel hierarchical inference learning approach that models local inference correctness probability, proposing policies with optimal regret bounds and low computational complexity, improving edge inference efficiency.

Contribution

The paper presents a new HIL method using confidence-based probability modeling and UCB policies, achieving order-optimal regret and low complexity for resource-constrained devices.

Findings

Both policies achieve $O( ext{log} T)$ regret.

HI-LCB-lite has $O(1)$ per-sample complexity.

Simulations show outperforming state-of-the-art methods.

Abstract

This work focuses on Hierarchical Inference (HI) in edge intelligence systems, where a compact Local-ML model on an end-device works in conjunction with a high-accuracy Remote-ML model on an edge-server. HI aims to reduce latency, improve accuracy, and lower bandwidth usage by first using the Local-ML model for inference and offloading to the Remote-ML only when the local inference is likely incorrect. A critical challenge in HI is estimating the likelihood of the local inference being incorrect, especially when data distributions and offloading costs change over time -- a problem we term Hierarchical Inference Learning (HIL). We introduce a novel approach to HIL by modeling the probability of correct inference by the Local-ML as an increasing function of the model's confidence measure, a structure motivated by empirical observations but previously unexploited. We propose two policies, HI-LCB and HI-LCB-lite, based on the Upper Confidence Bound (UCB) framework. We demonstrate that both policies achieve order-optimal regret of $O(\log T)$, a significant improvement over existing HIL policies with $O(T^{2/3})$ regret guarantees. Notably, HI-LCB-lite has an $O(1)$ per-sample computational complexity, making it well-suited for deployment on devices with severe resource limitations. Simulations using real-world datasets confirm that our policies outperform existing state-of-the-art HIL methods.