ADKO: Agentic Decentralized Knowledge Optimization

TL;DR

ADKO introduces a decentralized framework for collaborative black-box optimization that balances privacy, communication efficiency, and heterogeneity, validated through neural architecture search and scientific discovery experiments.

Contribution

It unifies multiple optimization paradigms with a formal analysis of information loss and proposes fidelity-aware token pruning for efficient knowledge sharing.

Findings

ADKO achieves sample efficiency and privacy preservation in decentralized optimization.

Theoretical analysis decomposes regret into GP error, LM bias/noise, and compression loss.

Experiments demonstrate consistent improvements over strong baselines.

Abstract

We present Agentic Decentralized Knowledge Optimization (ADKO), a framework for collaborative black-box optimization across autonomous agents that achieves sample efficiency, privacy preservation, heterogeneous-objective handling, and communication efficiency. Each agent maintains a private Gaussian Process (GP) surrogate trained on local data and communicates only through knowledge tokens-compact, lossy summaries containing directional signals, advantage scores, and optional language-model (LM) insights-without sharing raw data or model parameters. ADKO unifies GP-Upper Confidence Bound (GP-UCB), parallel Bayesian optimization, decentralized learning, and LM-guided discovery. We provide the first formal analysis of dual information loss: token compression, quantified via mutual-information-based fidelity, and LM approximation error, decomposed into bias and stochastic noise. Our main result shows cumulative regret decomposes into GP error, LM bias, LM noise, and compression loss, with necessary and sufficient conditions for sublinear regret. We also propose fidelity-aware token pruning to preserve high-information tokens under memory budget. Experiments on neural architecture search and scientific discovery validate the theory and show consistent improvements over strong baselines.