Self-Play Reinforcement Learning for Fast Image Retargeting

TL;DR

This paper introduces a reinforcement learning approach with self-play for efficient image retargeting, significantly reducing processing time while maintaining high retargeting quality.

Contribution

It proposes a novel RL-based method with a self-play reward and dynamic loss weighting to predict optimal retargeting operators efficiently.

Findings

Achieved 1000x faster retargeting processing time.

Maintained retargeting quality comparable to state-of-the-art methods.

Reduced processing time without sacrificing image quality.

Abstract

In this study, we address image retargeting, which is a task that adjusts input images to arbitrary sizes. In one of the best-performing methods called MULTIOP, multiple retargeting operators were combined and retargeted images at each stage were generated to find the optimal sequence of operators that minimized the distance between original and retargeted images. The limitation of this method is in its tremendous processing time, which severely prohibits its practical use. Therefore, the purpose of this study is to find the optimal combination of operators within a reasonable processing time; we propose a method of predicting the optimal operator for each step using a reinforcement learning agent. The technical contributions of this study are as follows. Firstly, we propose a reward based on self-play, which will be insensitive to the large variance in the content-dependent distance measured in MULTIOP. Secondly, we propose to dynamically change the loss weight for each action to prevent the algorithm from falling into a local optimum and from choosing only the most frequently used operator in its training. Our experiments showed that we achieved multi-operator image retargeting with less processing time by three orders of magnitude and the same quality as the original multi-operator-based method, which was the best-performing algorithm in retargeting tasks.