DCAN: Improving Temporal Action Detection via Dual Context Aggregation

TL;DR

DCAN introduces a dual context aggregation approach at boundary and proposal levels, significantly enhancing temporal action detection accuracy by capturing long-range and semantic context, achieving state-of-the-art results.

Contribution

The paper proposes the Dual Context Aggregation Network (DCAN), a novel end-to-end method that improves temporal action detection by aggregating context at multiple levels for better proposal quality.

Findings

Achieves 35.39% mAP on ActivityNet v1.3

Reaches 54.14% mAP at IoU@0.5 on THUMOS-14

Outperforms previous state-of-the-art methods

Abstract

Temporal action detection aims to locate the boundaries of action in the video. The current method based on boundary matching enumerates and calculates all possible boundary matchings to generate proposals. However, these methods neglect the long-range context aggregation in boundary prediction. At the same time, due to the similar semantics of adjacent matchings, local semantic aggregation of densely-generated matchings cannot improve semantic richness and discrimination. In this paper, we propose the end-to-end proposal generation method named Dual Context Aggregation Network (DCAN) to aggregate context on two levels, namely, boundary level and proposal level, for generating high-quality action proposals, thereby improving the performance of temporal action detection. Specifically, we design the Multi-Path Temporal Context Aggregation (MTCA) to achieve smooth context aggregation on boundary level and precise evaluation of boundaries. For matching evaluation, Coarse-to-fine Matching (CFM) is designed to aggregate context on the proposal level and refine the matching map from coarse to fine. We conduct extensive experiments on ActivityNet v1.3 and THUMOS-14. DCAN obtains an average mAP of 35.39% on ActivityNet v1.3 and reaches mAP 54.14% at IoU@0.5 on THUMOS-14, which demonstrates DCAN can generate high-quality proposals and achieve state-of-the-art performance. We release the code at https://github.com/cg1177/DCAN.