Universal Semantic Segmentation for Fisheye Urban Driving Images

TL;DR

This paper introduces a seven degrees of freedom augmentation method to transform rectilinear images into fisheye images, enhancing semantic segmentation accuracy and robustness for autonomous driving applications with fisheye cameras.

Contribution

The paper presents a novel seven-DoF augmentation technique that enables universal semantic segmentation for fisheye images in autonomous driving, addressing dataset scarcity and distortion challenges.

Findings

Improved segmentation accuracy on fisheye images.

Enhanced robustness against different fisheye distortions.

Successful application on real-world fisheye data.

Abstract

Semantic segmentation is a critical method in the field of autonomous driving. When performing semantic image segmentation, a wider field of view (FoV) helps to obtain more information about the surrounding environment, making automatic driving safer and more reliable, which could be offered by fisheye cameras. However, large public fisheye datasets are not available, and the fisheye images captured by the fisheye camera with large FoV comes with large distortion, so commonly-used semantic segmentation model cannot be directly utilized. In this paper, a seven degrees of freedom (DoF) augmentation method is proposed to transform rectilinear image to fisheye image in a more comprehensive way. In the training process, rectilinear images are transformed into fisheye images in seven DoF, which simulates the fisheye images taken by cameras of different positions, orientations and focal lengths. The result shows that training with the seven-DoF augmentation can improve the model's accuracy and robustness against different distorted fisheye data. This seven-DoF augmentation provides a universal semantic segmentation solution for fisheye cameras in different autonomous driving applications. Also, we provide specific parameter settings of the augmentation for autonomous driving. At last, we tested our universal semantic segmentation model on real fisheye images and obtained satisfactory results. The code and configurations are released at https://github.com/Yaozhuwa/FisheyeSeg.