Introducing a Class-Aware Metric for Monocular Depth Estimation: An Automotive Perspective

TL;DR

This paper introduces a novel class-aware metric for monocular depth estimation tailored for automotive safety, providing more detailed insights into model performance, especially for critical and unseen classes.

Contribution

The paper proposes a new evaluation metric that incorporates class-wise, edge, corner, and global consistency components, weighted by scene distance and criticality, enhancing depth model assessment.

Findings

The new metric offers deeper insights compared to classical metrics.

It effectively identifies critical and safety-relevant situations.

The approach improves understanding of model performance in automotive scenarios.

Abstract

The increasing accuracy reports of metric monocular depth estimation models lead to a growing interest from the automotive domain. Current model evaluations do not provide deeper insights into the models' performance, also in relation to safety-critical or unseen classes. Within this paper, we present a novel approach for the evaluation of depth estimation models. Our proposed metric leverages three components, a class-wise component, an edge and corner image feature component, and a global consistency retaining component. Classes are further weighted on their distance in the scene and on criticality for automotive applications. In the evaluation, we present the benefits of our metric through comparison to classical metrics, class-wise analytics, and the retrieval of critical situations. The results show that our metric provides deeper insights into model results while fulfilling safety-critical requirements. We release the code and weights on the following repository: https://github.com/leisemann/ca_mmde