Collaborative Inference for Efficient Remote Monitoring

TL;DR

This paper introduces a collaborative inference framework that decomposes complex models into simple local monitoring functions and correction terms, enabling efficient and safe remote monitoring on edge devices.

Contribution

It proposes a novel model decomposition approach that balances accuracy and safety for remote monitoring, with theoretical analysis and practical validation.

Findings

Reduced model complexity with minimal safety violations

Effective early warning capabilities on edge devices

Framework applicable to cost-sensitive classification tasks

Abstract

While current machine learning models have impressive performance over a wide range of applications, their large size and complexity render them unsuitable for tasks such as remote monitoring on edge devices with limited storage and computational power. A naive approach to resolve this on the model level is to use simpler architectures, but this sacrifices prediction accuracy and is unsuitable for monitoring applications requiring accurate detection of the onset of adverse events. In this paper, we propose an alternative solution to this problem by decomposing the predictive model as the sum of a simple function which serves as a local monitoring tool, and a complex correction term to be evaluated on the server. A sign requirement is imposed on the latter to ensure that the local monitoring function is safe, in the sense that it can effectively serve as an early warning system. Our analysis quantifies the trade-offs between model complexity and performance, and serves as a guidance for architecture design. We validate our proposed framework on a series of monitoring experiments, where we succeed at learning monitoring models with significantly reduced complexity that minimally violate the safety requirement. More broadly, our framework is useful for learning classifiers in applications where false negatives are significantly more costly compared to false positives.