Hybrid Edge-HPC Systems for Low-Latency Data-Driven Inference

TL;DR

This paper introduces RBF, a hybrid edge-HPC architecture that enables low-latency inference with asynchronous model updates driven by high-fidelity simulations, demonstrated in a digital agriculture setting.

Contribution

The paper presents RBF, a novel hybrid system architecture that decouples inference from simulation-driven model updates across heterogeneous infrastructure.

Findings

RBF achieves continuous low-latency inference despite delayed model updates.

The system improves model fidelity over time through opportunistic HPC backfilling.

Evaluation shows effective integration of edge inference with simulation-based model refinement.

Abstract

Emerging cyber-physical systems increasingly require low-latency inference from streaming sensor data while maintaining models that reflect complex and evolving physical processes. In many domains, however, model updates depend on high-fidelity simulations and training executed on remote high-performance computing (HPC) systems under batch scheduling. This creates a fundamental mismatch between the responsiveness required at the edge and the cost, throughput, and availability of simulation-driven model updates. We present RBF (Reverse Backfill), a hybrid edge-HPC learning and inference architecture that integrates low-latency edge inference with asynchronous, simulation-driven model improvement. RBF targets simulation-bounded settings in which model updates are constrained by simulation throughput and HPC scheduling delays, and reinterprets HPC backfilling by using opportunistic computation to improve model accuracy rather than system utilization. RBF decouples inference from simulation and training by deploying lightweight surrogate models at the edge while incorporating improved models asynchronously as they become available. The architecture supports pluggable surrogate models and orchestrates computation across heterogeneous infrastructure spanning edge devices, private 5G, cloud, and HPC resources. We instantiate RBF using a real-world digital agriculture deployment that couples edge sensing with computational fluid dynamics (CFD) simulations to infer airflow patterns in a large agricultural screenhouse. Our evaluation characterizes end-to-end system behavior under realistic constraints, quantifying simulation latency, training cost, inference throughput, and the impact of delayed model updates on prediction accuracy. Results demonstrate that RBF enables continuous, low-latency inference while improving model fidelity over time despite delayed and irregular model updates.