Rethinking Camera Choice: An Empirical Study on Fisheye Camera Properties in Robotic Manipulation

TL;DR

This study systematically investigates the effects of wrist-mounted fisheye cameras on robotic imitation learning, revealing their advantages and limitations in spatial localization, scene, and hardware generalization, and proposing a simple augmentation method for improvement.

Contribution

It provides the first comprehensive empirical analysis of fisheye camera properties in robotic manipulation, offering practical insights and a simple augmentation strategy for better generalization.

Findings

Wide FoV improves spatial localization in complex environments.

Fisheye-trained policies generalize better with diverse training environments.

Random Scale Augmentation enhances cross-camera hardware generalization.

Abstract

The adoption of fisheye cameras in robotic manipulation, driven by their exceptionally wide Field of View (FoV), is rapidly outpacing a systematic understanding of their downstream effects on policy learning. This paper presents the first comprehensive empirical study to bridge this gap, rigorously analyzing the properties of wrist-mounted fisheye cameras for imitation learning. Through extensive experiments in both simulation and the real world, we investigate three critical research questions: spatial localization, scene generalization, and hardware generalization. Our investigation reveals that: (1) The wide FoV significantly enhances spatial localization, but this benefit is critically contingent on the visual complexity of the environment. (2) Fisheye-trained policies, while prone to overfitting in simple scenes, unlock superior scene generalization when trained with sufficient environmental diversity. (3) While naive cross-camera transfer leads to failures, we identify the root cause as scale overfitting and demonstrate that hardware generalization performance can be improved with a simple Random Scale Augmentation (RSA) strategy. Collectively, our findings provide concrete, actionable guidance for the large-scale collection and effective use of fisheye datasets in robotic learning. More results and videos are available on https://robo-fisheye.github.io/