Linearly-evolved Transformer for Pan-sharpening

TL;DR

This paper introduces a lightweight, linearly-evolved transformer for satellite pan-sharpening that maintains high performance while significantly reducing computational costs, making it suitable for low-resource satellite applications.

Contribution

It develops a novel 1-order linearly-evolved transformer variant that offers an efficient alternative to traditional cascaded transformers in pan-sharpening tasks.

Findings

Achieves competitive performance with fewer computational resources.

Demonstrates effectiveness on multiple satellite datasets.

Validates generalization to hyper-spectral image fusion.

Abstract

Vision transformer family has dominated the satellite pan-sharpening field driven by the global-wise spatial information modeling mechanism from the core self-attention ingredient. The standard modeling rules within these promising pan-sharpening methods are to roughly stack the transformer variants in a cascaded manner. Despite the remarkable advancement, their success may be at the huge cost of model parameters and FLOPs, thus preventing its application over low-resource satellites.To address this challenge between favorable performance and expensive computation, we tailor an efficient linearly-evolved transformer variant and employ it to construct a lightweight pan-sharpening framework. In detail, we deepen into the popular cascaded transformer modeling with cutting-edge methods and develop the alternative 1-order linearly-evolved transformer variant with the 1-dimensional linear convolution chain to achieve the same function. In this way, our proposed method is capable of benefiting the cascaded modeling rule while achieving favorable performance in the efficient manner. Extensive experiments over multiple satellite datasets suggest that our proposed method achieves competitive performance against other state-of-the-art with fewer computational resources. Further, the consistently favorable performance has been verified over the hyper-spectral image fusion task. Our main focus is to provide an alternative global modeling framework with an efficient structure. The code will be publicly available.