# Reinforcement Learning with Depreciating Assets

## TL;DR

This paper explores reinforcement learning where rewards, termed assets, depreciate over time, introducing a new framework that accounts for reward decay and proposing a model-free method to find optimal policies.

## Contribution

It introduces the concept of depreciating assets in reinforcement learning, extending traditional models to include reward decay over time with a Bellman-style optimality framework.

## Key findings

- Developed a Bellman-style equation for assets with decay.
- Proposed a model-free reinforcement learning algorithm for this setting.
- Demonstrated the effectiveness of the approach in theoretical scenarios.

## Abstract

A basic assumption of traditional reinforcement learning is that the value of a reward does not change once it is received by an agent. The present work forgoes this assumption and considers the situation where the value of a reward decays proportionally to the time elapsed since it was obtained. Emphasizing the inflection point occurring at the time of payment, we use the term asset to refer to a reward that is currently in the possession of an agent. Adopting this language, we initiate the study of depreciating assets within the framework of infinite-horizon quantitative optimization. In particular, we propose a notion of asset depreciation, inspired by classical exponential discounting, where the value of an asset is scaled by a fixed discount factor at each time step after it is obtained by the agent. We formulate a Bellman-style equational characterization of optimality in this context and develop a model-free reinforcement learning approach to obtain optimal policies.