Tree-Structured Reinforcement Learning for Sequential Object Localization

TL;DR

This paper introduces Tree-RL, a reinforcement learning method that sequentially searches for multiple objects in images by exploiting interdependencies and historical search paths, improving efficiency and diversity over existing algorithms.

Contribution

The paper proposes a novel Tree-RL approach that models object localization as a sequential, tree-structured search process, capturing inter-object dependencies and enabling multiple object discovery with fewer proposals.

Findings

Achieves comparable recall with fewer candidate windows.

Effectively models inter-object dependencies.

Provides diverse search paths for multiple object detection.

Abstract

Existing object proposal algorithms usually search for possible object regions over multiple locations and scales separately, which ignore the interdependency among different objects and deviate from the human perception procedure. To incorporate global interdependency between objects into object localization, we propose an effective Tree-structured Reinforcement Learning (Tree-RL) approach to sequentially search for objects by fully exploiting both the current observation and historical search paths. The Tree-RL approach learns multiple searching policies through maximizing the long-term reward that reflects localization accuracies over all the objects. Starting with taking the entire image as a proposal, the Tree-RL approach allows the agent to sequentially discover multiple objects via a tree-structured traversing scheme. Allowing multiple near-optimal policies, Tree-RL offers more diversity in search paths and is able to find multiple objects with a single feed-forward pass. Therefore, Tree-RL can better cover different objects with various scales which is quite appealing in the context of object proposal. Experiments on PASCAL VOC 2007 and 2012 validate the effectiveness of the Tree-RL, which can achieve comparable recalls with current object proposal algorithms via much fewer candidate windows.