MSDS: Deep Structural Similarity with Multiscale Representation

TL;DR

This paper introduces MSDS, a multiscale extension of DeepSSIM, which improves deep perceptual similarity modeling by explicitly incorporating spatial scale, leading to better alignment with human visual perception.

Contribution

The paper presents a minimal multiscale framework that isolates and demonstrates the importance of spatial scale in deep-feature-based perceptual similarity models.

Findings

MSDS outperforms single-scale DeepSSIM on benchmark datasets.

Incorporating multiscale information yields statistically significant improvements.

The approach introduces negligible additional complexity.

Abstract

Deep-feature-based perceptual similarity models have demonstrated strong alignment with human visual perception in Image Quality Assessment (IQA). However, most existing approaches operate at a single spatial scale, implicitly assuming that structural similarity at a fixed resolution is sufficient. The role of spatial scale in deep-feature similarity modeling thus remains insufficiently understood. In this letter, we isolate spatial scale as an independent factor using a minimal multiscale extension of DeepSSIM, referred to as Deep Structural Similarity with Multiscale Representation (MSDS). The proposed framework decouples deep feature representation from cross-scale integration by computing DeepSSIM independently across pyramid levels and fusing the resulting scores with a lightweight set of learnable global weights. Experiments on multiple benchmark datasets demonstrate consistent and statistically significant improvements over the single-scale baseline, while introducing negligible additional complexity. The results empirically confirm spatial scale as a non-negligible factor in deep perceptual similarity, isolated here via a minimal testbed.