A Label-Free and Non-Monotonic Metric for Evaluating Denoising in Event Cameras

TL;DR

This paper introduces AOCC, a novel label-free, non-monotonic metric for evaluating event camera denoising by measuring contrast curve areas, addressing limitations of existing label-dependent and monotonic metrics.

Contribution

The paper presents the first non-monotonic, label-free evaluation metric for event camera denoising, based on contrast curve analysis, validated through theoretical and experimental results.

Findings

AOCC effectively evaluates denoising without labels.

The metric captures edge contours preserved during denoising.

Validation shows AOCC aligns with denoising quality.

Abstract

Event cameras are renowned for their high efficiency due to outputting a sparse, asynchronous stream of events. However, they are plagued by noisy events, especially in low light conditions. Denoising is an essential task for event cameras, but evaluating denoising performance is challenging. Label-dependent denoising metrics involve artificially adding noise to clean sequences, complicating evaluations. Moreover, the majority of these metrics are monotonic, which can inflate scores by removing substantial noise and valid events. To overcome these limitations, we propose the first label-free and non-monotonic evaluation metric, the area of the continuous contrast curve (AOCC), which utilizes the area enclosed by event frame contrast curves across different time intervals. This metric is inspired by how events capture the edge contours of scenes or objects with high temporal resolution. An effective denoising method removes noise without eliminating these edge-contour events, thus preserving the contrast of event frames. Consequently, contrast across various time ranges serves as a metric to assess denoising effectiveness. As the time interval lengthens, the curve will initially rise and then fall. The proposed metric is validated through both theoretical and experimental evidence.