Structure-Regularized Attention for Deformable Object Representation

TL;DR

This paper introduces a structure-regularized attention mechanism that enhances deformable object representation by modeling intrinsic structural dependencies, improving performance and efficiency in neural networks.

Contribution

It proposes a novel structure-regularized attention method that formalizes feature interactions via structural factorization, seamlessly integrating into CNNs for better deformable object modeling.

Findings

Improves performance across multiple tasks.

Reduces model complexity compared to existing attention methods.

Captures diversified object part representations without extra supervision.

Abstract

Capturing contextual dependencies has proven useful to improve the representational power of deep neural networks. Recent approaches that focus on modeling global context, such as self-attention and non-local operation, achieve this goal by enabling unconstrained pairwise interactions between elements. In this work, we consider learning representations for deformable objects which can benefit from context exploitation by modeling the structural dependencies that the data intrinsically possesses. To this end, we provide a novel structure-regularized attention mechanism, which formalizes feature interaction as structural factorization through the use of a pair of light-weight operations. The instantiated building blocks can be directly incorporated into modern convolutional neural networks, to boost the representational power in an efficient manner. Comprehensive studies on multiple tasks and empirical comparisons with modern attention mechanisms demonstrate the gains brought by our method in terms of both performance and model complexity. We further investigate its effect on feature representations, showing that our trained models can capture diversified representations characterizing object parts without resorting to extra supervision.