IslandRun: Privacy-Aware Multi-Objective Orchestration for Distributed AI Inference

TL;DR

IslandRun is a novel multi-objective orchestration system for distributed AI inference that balances performance, privacy, and cost across heterogeneous resources by leveraging data locality, trust tiers, and privacy-preserving routing.

Contribution

It introduces a new decentralized orchestration paradigm with agent-based routing, trust-aware island grouping, and reversible anonymization for privacy-aware AI inference.

Findings

Effective multi-objective optimization across heterogeneous resources

Data locality-based routing improves privacy and efficiency

Privacy-preserving techniques enable trust boundary management

Abstract

Modern AI inference faces an irreducible tension: no single computational resource simultaneously maximizes performance, preserves privacy, minimizes cost, and maintains trust. Existing orchestration frameworks optimize single dimensions (Kubernetes prioritizes latency, federated learning preserves privacy, edge computing reduces network distance), creating solutions that struggle under real-world heterogeneity. We present IslandRun, a multi-objective orchestration system that treats computational resources as autonomous "islands" spanning personal devices, private edge servers, and public cloud. Our key insights: (1) request-level heterogeneity demands policy-constrained multi-objective optimization, (2) data locality enables routing compute to data rather than data to compute, and (3) typed placeholder sanitization preserves context semantics across trust boundaries. IslandRun introduces agent-based routing, tiered island groups with differential trust, and reversible anonymization. This establishes a new paradigm for privacy-aware, decentralized inference orchestration across heterogeneous personal computing ecosystems.