Cooperative Saliency-based Obstacle Detection and AR Rendering for Increased Situational Awareness

TL;DR

This paper presents a cooperative system combining saliency-based obstacle detection with augmented reality visualization to enhance driver awareness and safety in autonomous driving scenarios.

Contribution

It introduces a novel distributed cooperative obstacle detection and AR rendering scheme leveraging sensing, communication, and saliency techniques.

Findings

Favorable detection results on pothole datasets.

Effective AR visualization of hazards.

Improved situational awareness in simulated tests.

Abstract

Autonomous vehicles are expected to operate safely in real-life road conditions in the next years. Nevertheless, unanticipated events such as the existence of unexpected objects in the range of the road, can put safety at risk. The advancement of sensing and communication technologies and Internet of Things may facilitate the recognition of hazardous situations and information exchange in a cooperative driving scheme, providing new opportunities for the increase of collaborative situational awareness. Safe and unobtrusive visualization of the obtained information may nowadays be enabled through the adoption of novel Augmented Reality (AR) interfaces in the form of windshields. Motivated by these technological opportunities, we propose in this work a saliency-based distributed, cooperative obstacle detection and rendering scheme for increasing the driver's situational awareness through (i) automated obstacle detection, (ii) AR visualization and (iii) information sharing (upcoming potential dangers) with other connected vehicles or road infrastructure. An extensive evaluation study using a variety of real datasets for pothole detection showed that the proposed method provides favorable results and features compared to other recent and relevant approaches.