Deep Learning for Omnidirectional Vision: A Survey and New Perspectives

TL;DR

This survey reviews recent deep learning techniques for omnidirectional vision, highlighting their principles, methods, applications, challenges, and future research directions in a comprehensive manner.

Contribution

It provides the first systematic taxonomy and analysis of deep learning approaches specifically designed for omnidirectional vision tasks.

Findings

Deep learning methods improve omnidirectional image processing.

New datasets and convolution techniques address unique ODI challenges.

Identifies open problems and future research directions in the field.

Abstract

Omnidirectional image (ODI) data is captured with a 360x180 field-of-view, which is much wider than the pinhole cameras and contains richer spatial information than the conventional planar images. Accordingly, omnidirectional vision has attracted booming attention due to its more advantageous performance in numerous applications, such as autonomous driving and virtual reality. In recent years, the availability of customer-level 360 cameras has made omnidirectional vision more popular, and the advance of deep learning (DL) has significantly sparked its research and applications. This paper presents a systematic and comprehensive review and analysis of the recent progress in DL methods for omnidirectional vision. Our work covers four main contents: (i) An introduction to the principle of omnidirectional imaging, the convolution methods on the ODI, and datasets to highlight the differences and difficulties compared with the 2D planar image data; (ii) A structural and hierarchical taxonomy of the DL methods for omnidirectional vision; (iii) A summarization of the latest novel learning strategies and applications; (iv) An insightful discussion of the challenges and open problems by highlighting the potential research directions to trigger more research in the community.