Rethinking Learnable Tree Filter for Generic Feature Transform

TL;DR

This paper introduces an improved learnable tree filter that relaxes geometric constraints, enabling better long-range dependency modeling and structural detail preservation in various vision tasks, with significant empirical performance gains.

Contribution

It reformulates the learnable tree filter as a Markov Random Field with a learnable unary term and proposes a differentiable spanning tree algorithm, enhancing flexibility and robustness.

Findings

Achieves 82.1% mIoU on Cityscapes semantic segmentation

Demonstrates consistent improvements in object detection and instance segmentation

Extends the method to multiple vision tasks with linear complexity

Abstract

The Learnable Tree Filter presents a remarkable approach to model structure-preserving relations for semantic segmentation. Nevertheless, the intrinsic geometric constraint forces it to focus on the regions with close spatial distance, hindering the effective long-range interactions. To relax the geometric constraint, we give the analysis by reformulating it as a Markov Random Field and introduce a learnable unary term. Besides, we propose a learnable spanning tree algorithm to replace the original non-differentiable one, which further improves the flexibility and robustness. With the above improvements, our method can better capture long-range dependencies and preserve structural details with linear complexity, which is extended to several vision tasks for more generic feature transform. Extensive experiments on object detection/instance segmentation demonstrate the consistent improvements over the original version. For semantic segmentation, we achieve leading performance (82.1% mIoU) on the Cityscapes benchmark without bells-and-whistles. Code is available at https://github.com/StevenGrove/LearnableTreeFilterV2.