MEL: Multi-level Ensemble Learning for Resource-Constrained Environments

TL;DR

MEL introduces a multi-level ensemble learning framework that enhances resilience and maintains high accuracy for resource-constrained edge inference by training lightweight, collaborative backup models.

Contribution

It proposes a novel multi-objective optimization approach to train diverse, lightweight models that collaboratively and independently ensure fault tolerance in edge environments.

Findings

Achieves similar accuracy to original models with 40% size.

Maintains 95.6% ensemble accuracy during failures.

Provides flexible deployment across various edge platforms.

Abstract

AI inference at the edge is becoming increasingly common for low-latency services. However, edge environments are power- and resource-constrained, and susceptible to failures. Conventional failure resilience approaches, such as cloud failover or compressed backups, often compromise latency or accuracy, limiting their effectiveness for critical edge inference services. In this paper, we propose Multi-Level Ensemble Learning (MEL), a new framework for resilient edge inference that simultaneously trains multiple lightweight backup models capable of operating collaboratively, refining each other when multiple servers are available, and independently under failures while maintaining good accuracy. Specifically, we formulate our approach as a multi-objective optimization problem with a loss formulation that inherently encourages diversity among individual models to promote mutually refining representations, while ensuring each model maintains good standalone performance. Empirical evaluations across vision, language, and audio datasets show that MEL provides performance comparable to original architectures while also providing fault tolerance and deployment flexibility across edge platforms. Our results show that our ensemble model, sized at 40\% of the original model, achieves similar performance, while preserving 95.6\% of ensemble accuracy in the case of failures when trained using MEL.