# A Micro-Objective Perspective of Reinforcement Learning

## TL;DR

This paper introduces micro-objective reinforcement learning, a new formalism that considers multiple objectives and prior knowledge, addressing limitations of traditional RL focused solely on expected reward.

## Contribution

It proposes a novel RL framework incorporating micro-objectives and temporal abstraction, broadening the scope of RL applications and theoretical understanding.

## Key findings

- Formalism unifies single and multi-objective RL
- Allows incorporation of prior knowledge through temporal abstraction
- Addresses distributional performance concerns in RL

## Abstract

The standard reinforcement learning (RL) formulation considers the expectation of the (discounted) cumulative reward. This is limiting in applications where we are concerned with not only the expected performance, but also the distribution of the performance. In this paper, we introduce micro-objective reinforcement learning --- an alternative RL formalism that overcomes this issue. In this new formulation, a RL task is specified by a set of micro-objectives, which are constructs that specify the desirability or undesirability of events. In addition, micro-objectives allow prior knowledge in the form of temporal abstraction to be incorporated into the global RL objective. The generality of this formalism, and its relations to single/multi-objective RL, and hierarchical RL are discussed.