Differentiable Weightless Controllers: Learning Logic Circuits for Continuous Control

TL;DR

This paper introduces Differentiable Weightless Controllers (DWCs), a novel symbolic-differentiable architecture that learns logic circuit-based policies for continuous control, enabling FPGA deployment with high efficiency and interpretability.

Contribution

The paper presents DWCs, a new architecture that learns logic circuit policies end-to-end with gradient methods, suitable for fast, energy-efficient FPGA implementation, and offers interpretability.

Findings

DWCs achieve competitive performance on MuJoCo benchmarks.

DWCs can be directly compiled into FPGA-compatible circuits.

DWCs exhibit sparse, interpretable connectivity patterns.

Abstract

We investigate whether continuous-control policies can be represented and learned as discrete logic circuits instead of continuous neural networks. We introduce Differentiable Weightless Controllers (DWCs), a symbolic-differentiable architecture that maps real-valued observations to actions using thermometer-encoded inputs, sparsely connected boolean lookup-table layers, and lightweight action heads. DWCs can be trained end-to-end by gradient-based techniques, yet compile directly into FPGA-compatible circuits with few- or even single-clock-cycle latency and nanojoule-level energy cost per action. Across five MuJoCo benchmarks, including high-dimensional Humanoid, DWCs achieve returns competitive with weight-based policies (full precision or quantized neural networks), matching performance on four tasks and isolating network capacity as the key limiting factor on HalfCheetah. Furthermore, DWCs exhibit structurally sparse and interpretable connectivity patterns, enabling a direct inspection of which input thresholds influence control decisions.