Risk-sensitive Actor-free Policy via Convex Optimization

TL;DR

This paper introduces a risk-sensitive, actor-free reinforcement learning policy using convex optimization and neural networks to ensure safety and global optimality in decision-making.

Contribution

It proposes a novel actor-free policy framework that employs input-convex neural networks for risk-sensitive optimization in reinforcement learning.

Findings

Effective risk control demonstrated in experiments

Convex neural network ensures global optimality

Simplifies policy optimization process

Abstract

Traditional reinforcement learning methods optimize agents without considering safety, potentially resulting in unintended consequences. In this paper, we propose an optimal actor-free policy that optimizes a risk-sensitive criterion based on the conditional value at risk. The risk-sensitive objective function is modeled using an input-convex neural network ensuring convexity with respect to the actions and enabling the identification of globally optimal actions through simple gradient-following methods. Experimental results demonstrate the efficacy of our approach in maintaining effective risk control.