Remote Action Generation: Remote Control with Minimal Communication

TL;DR

This paper introduces GRASP, a framework for remote control that minimizes communication by enabling actors to generate actions locally through guided sampling, significantly reducing data transmission.

Contribution

The paper proposes a novel remote generation framework that reduces communication in remote control by allowing actors to locally generate actions via guided sampling.

Findings

Achieves an average 12-fold reduction in communication data.

Demonstrates 50-fold reduction for continuous action spaces.

Reduces communication by 41-fold compared to transmitting rewards.

Abstract

We address the challenge of remote control where one or more actors, lacking direct reward access, are steered by a controller over a communication-constrained channel. The controller learns an optimal policy from observed rewards and communicates action guidance to the actors, which becomes demanding for large or continuous action spaces. To achieve rate-efficient communication throughout this interactive learning and control process, we introduce a novel framework leveraging remote generation. Instead of transmitting full action specifications, the controller sends minimal information, enabling the actors to locally generate actions by sampling from the controller's evolving target policy. This guided sampling is facilitated by an importance sampling approach. Concurrently, the actors use the received guidance as supervised learning data to learn the controller's policy. This actor-side learning improves their local sampling capabilities, progressively reducing future communication needs. Our solution, Guided Remote Action Sampling Policy (GRASP), demonstrates significant communication reduction, achieving an average 12-fold data reduction across all experiments (50-fold for continuous action spaces) compared to direct action transmission, and a 41-fold reduction compared to reward transmission.