SALAD: Self-Assessment Learning for Action Detection

TL;DR

This paper introduces SALAD, a self-assessment learning method for action detection that improves localization accuracy by integrating confidence estimation, outperforming state-of-the-art benchmarks.

Contribution

It presents a novel self-assessment learning framework that enhances action detection performance by leveraging confidence scores as a regularizer.

Findings

Outperforms state-of-the-art on THUMOS14 with 44.6% mAP at tIoU@0.5

Achieves 51.7% mAP on ActivityNet1.3 at tIoU@0.5

Significant improvements at lower tIoU thresholds

Abstract

Literature on self-assessment in machine learning mainly focuses on the production of well-calibrated algorithms through consensus frameworks i.e. calibration is seen as a problem. Yet, we observe that learning to be properly confident could behave like a powerful regularization and thus, could be an opportunity to improve performance.Precisely, we show that used within a framework of action detection, the learning of a self-assessment score is able to improve the whole action localization process.Experimental results show that our approach outperforms the state-of-the-art on two action detection benchmarks. On THUMOS14 dataset, the mAP at tIoU@0.5 is improved from 42.8\% to 44.6\%, and from 50.4\% to 51.7\% on ActivityNet1.3 dataset. For lower tIoU values, we achieve even more significant improvements on both datasets.