Deep reinforcement learning for the olfactory search POMDP: a quantitative benchmark

TL;DR

This paper benchmarks deep reinforcement learning against traditional POMDP solvers for olfactory search tasks, demonstrating its effectiveness in generating lightweight policies suitable for robotic applications.

Contribution

It provides a quantitative comparison showing deep reinforcement learning as a competitive approach for solving olfactory search POMDPs efficiently.

Findings

Deep RL produces lightweight policies for olfactory search.

Deep RL is competitive with traditional POMDP solvers.

The benchmark offers insights into approximate solutions for complex POMDPs.

Abstract

The olfactory search POMDP (partially observable Markov decision process) is a sequential decision-making problem designed to mimic the task faced by insects searching for a source of odor in turbulence, and its solutions have applications to sniffer robots. As exact solutions are out of reach, the challenge consists in finding the best possible approximate solutions while keeping the computational cost reasonable. We provide a quantitative benchmarking of a solver based on deep reinforcement learning against traditional POMDP approximate solvers. We show that deep reinforcement learning is a competitive alternative to standard methods, in particular to generate lightweight policies suitable for robots.