Data-Driven Structured Policy Iteration for Homogeneous Distributed Systems

TL;DR

This paper introduces D2SPI, a data-driven algorithm for designing structured, stabilizing feedback controllers in networked systems, using auxiliary links during learning to ensure stability and convergence.

Contribution

The paper presents a novel data-driven policy iteration method that respects network sparsity and guarantees stability and convergence during learning.

Findings

D2SPI successfully stabilizes networked systems in simulations.

The method ensures convergence of policies throughout the learning process.

Auxiliary links facilitate effective data exchange during policy synthesis.

Abstract

Control of networked systems, comprised of interacting agents, is often achieved through modeling the underlying interactions. Constructing accurate models of such interactions--in the meantime--can become prohibitive in applications. Data-driven control methods avoid such complications by directly synthesizing a controller from the observed data. In this paper, we propose an algorithm referred to as Data-driven Structured Policy Iteration (D2SPI), for synthesizing an efficient feedback mechanism that respects the sparsity pattern induced by the underlying interaction network. In particular, our algorithm uses temporary "auxiliary" communication links in order to enable the required information exchange on a (smaller) sub-network during the "learning phase" -- links that will be removed subsequently for the final distributed feedback synthesis. We then proceed to show that the learned policy results in a stabilizing structured policy for the entire network. Our analysis is then followed by showing the stability and convergence of the proposed distributed policies throughout the learning phase, exploiting a construct referred to as the "Patterned monoid.'' The performance of D2SPI is then demonstrated using representative simulation scenarios.