# Rotational Rectification Network: Enabling Pedestrian Detection for Mobile Vision

## TL;DR

This paper introduces R2N, a module that improves pedestrian detection in images with significant camera rotation by estimating and correcting for rotation, significantly enhancing detection accuracy on mobile and rugged terrain platforms.

## Contribution

The paper presents a novel Rotational Rectification Network with a Global Polar Pooling operator, enabling CNN-based detectors to adapt to large camera rotations, which was not addressed in prior work.

## Key findings

- Improves pedestrian detection accuracy under heavy image rotation by up to 45%.
- The R2N module can be integrated into existing CNN detectors without retraining from scratch.
- The GP-Pooling operator effectively captures rotational shifts in features.

## Abstract

Across a majority of pedestrian detection datasets, it is typically assumed that pedestrians will be standing upright with respect to the image coordinate system. This assumption, however, is not always valid for many vision-equipped mobile platforms such as mobile phones, UAVs or construction vehicles on rugged terrain. In these situations, the motion of the camera can cause images of pedestrians to be captured at extreme angles. This can lead to very poor pedestrian detection performance when using standard pedestrian detectors. To address this issue, we propose a Rotational Rectification Network (R2N) that can be inserted into any CNN-based pedestrian (or object) detector to adapt it to significant changes in camera rotation. The rotational rectification network uses a 2D rotation estimation module that passes rotational information to a spatial transformer network to undistort image features. To enable robust rotation estimation, we propose a Global Polar Pooling (GP-Pooling) operator to capture rotational shifts in convolutional features. Through our experiments, we show how our rotational rectification network can be used to improve the performance of the state-of-the-art pedestrian detector under heavy image rotation by up to 45%