A3-FPN: Asymptotic Content-Aware Pyramid Attention Network for Dense Visual Prediction

TL;DR

A3-FPN introduces an asymptotic content-aware pyramid attention network that enhances multi-scale feature representation for dense visual prediction, improving performance on benchmarks like MS COCO and Cityscapes.

Contribution

It proposes a novel asymptotically disentangled framework with content-aware attention modules to better capture discriminative features and small objects.

Findings

Achieves 49.6 mask AP on MS COCO with Swin-L backbone.

Improves Cityscapes mIoU to 85.6 with A3-FPN.

Demonstrates compatibility with CNN and Transformer architectures.

Abstract

Learning multi-scale representations is the common strategy to tackle object scale variation in dense prediction tasks. Although existing feature pyramid networks have greatly advanced visual recognition, inherent design defects inhibit them from capturing discriminative features and recognizing small objects. In this work, we propose Asymptotic Content-Aware Pyramid Attention Network (A3-FPN), to augment multi-scale feature representation via the asymptotically disentangled framework and content-aware attention modules. Specifically, A3-FPN employs a horizontally-spread column network that enables asymptotically global feature interaction and disentangles each level from all hierarchical representations. In feature fusion, it collects supplementary content from the adjacent level to generate position-wise offsets and weights for context-aware resampling, and learns deep context reweights to improve intra-category similarity. In feature reassembly, it further strengthens intra-scale discriminative feature learning and reassembles redundant features based on information content and spatial variation of feature maps. Extensive experiments on MS COCO, VisDrone2019-DET and Cityscapes demonstrate that A3-FPN can be easily integrated into state-of-the-art CNN and Transformer-based architectures, yielding remarkable performance gains. Notably, when paired with OneFormer and Swin-L backbone, A3-FPN achieves 49.6 mask AP on MS COCO and 85.6 mIoU on Cityscapes. Codes are available at https://github.com/mason-ching/A3-FPN.