CDRL: A Reinforcement Learning Framework Inspired by Cerebellar Circuits and Dendritic Computational Strategies

TL;DR

This paper introduces a biologically inspired reinforcement learning architecture based on cerebellar principles, enhancing sample efficiency, robustness, and generalization in high-dimensional noisy environments.

Contribution

It presents a novel RL framework incorporating cerebellar-inspired structural priors and dendritic modulation, improving performance over traditional designs.

Findings

Enhanced sample efficiency in noisy, high-dimensional tasks

Improved robustness and generalization with cerebellar architecture

Architectural priors optimize RL performance with fewer parameters

Abstract

Reinforcement learning (RL) has achieved notable performance in high-dimensional sequential decision-making tasks, yet remains limited by low sample efficiency, sensitivity to noise, and weak generalization under partial observability. Most existing approaches address these issues primarily through optimization strategies, while the role of architectural priors in shaping representation learning and decision dynamics is less explored. Inspired by structural principles of the cerebellum, we propose a biologically grounded RL architecture that incorporate large expansion, sparse connectivity, sparse activation, and dendritic-level modulation. Experiments on noisy, high-dimensional RL benchmarks show that both the cerebellar architecture and dendritic modulation consistently improve sample efficiency, robustness, and generalization compared to conventional designs. Sensitivity analysis of architectural parameters suggests that cerebellum-inspired structures can offer optimized performance for RL with constrained model parameters. Overall, our work underscores the value of cerebellar structural priors as effective inductive biases for RL.