# Flow-based Intrinsic Curiosity Module

## TL;DR

This paper introduces FICM, a flow-based intrinsic curiosity module that uses optical flow prediction errors to enhance exploration in deep reinforcement learning, especially in environments with moving objects.

## Contribution

The paper presents a novel flow-based curiosity module leveraging motion features for improved exploration in DRL, particularly in dynamic environments.

## Key findings

- FICM outperforms existing methods in environments with moving objects.
- FICM requires only two consecutive frames for effective novelty estimation.
- FICM is especially effective in environments like Atari, Super Mario Bros., and ViZDoom.

## Abstract

In this paper, we focus on a prediction-based novelty estimation strategy upon the deep reinforcement learning (DRL) framework, and present a flow-based intrinsic curiosity module (FICM) to exploit the prediction errors from optical flow estimation as exploration bonuses. We propose the concept of leveraging motion features captured between consecutive observations to evaluate the novelty of observations in an environment. FICM encourages a DRL agent to explore observations with unfamiliar motion features, and requires only two consecutive frames to obtain sufficient information when estimating the novelty. We evaluate our method and compare it with a number of existing methods on multiple benchmark environments, including Atari games, Super Mario Bros., and ViZDoom. We demonstrate that FICM is favorable to tasks or environments featuring moving objects, which allow FICM to utilize the motion features between consecutive observations. We further ablatively analyze the encoding efficiency of FICM, and discuss its applicable domains comprehensively.