Kolmogorov-Arnold Network for Online Reinforcement Learning

TL;DR

This paper investigates the integration of Kolmogorov-Arnold Networks into the PPO reinforcement learning framework, demonstrating comparable performance to traditional MLPs with fewer parameters on robotics benchmarks.

Contribution

It introduces the use of KANs as function approximators in PPO, showing their efficiency and potential advantages over MLPs in reinforcement learning.

Findings

KAN-based PPO achieves similar performance to MLP-based PPO.

KANs require fewer parameters, reducing memory usage.

The approach is effective on DeepMind Control Proprio Robotics benchmark.

Abstract

Kolmogorov-Arnold Networks (KANs) have shown potential as an alternative to Multi-Layer Perceptrons (MLPs) in neural networks, providing universal function approximation with fewer parameters and reduced memory usage. In this paper, we explore the use of KANs as function approximators within the Proximal Policy Optimization (PPO) algorithm. We evaluate this approach by comparing its performance to the original MLP-based PPO using the DeepMind Control Proprio Robotics benchmark. Our results indicate that the KAN-based reinforcement learning algorithm can achieve comparable performance to its MLP-based counterpart, often with fewer parameters. These findings suggest that KANs may offer a more efficient option for reinforcement learning models.