MS-Glance: Bio-Insipred Non-semantic Context Vectors and their Applications in Supervising Image Reconstruction

TL;DR

MS-Glance introduces biologically inspired non-semantic context vectors and a Glance Index for improved image reconstruction, demonstrating superior performance in natural and medical imaging tasks.

Contribution

The paper proposes MS-Glance, a novel non-semantic context descriptor inspired by human perception, and a Glance Index for image comparison, enhancing image reconstruction methods.

Findings

MS-Glance outperforms existing loss functions in image restoration.

Effective in both natural and medical image reconstruction.

Improves image fitting with implicit neural representations and MRI reconstruction.

Abstract

Non-semantic context information is crucial for visual recognition, as the human visual perception system first uses global statistics to process scenes rapidly before identifying specific objects. However, while semantic information is increasingly incorporated into computer vision tasks such as image reconstruction, non-semantic information, such as global spatial structures, is often overlooked. To bridge the gap, we propose a biologically informed non-semantic context descriptor, \textbf{MS-Glance}, along with the Glance Index Measure for comparing two images. A Global Glance vector is formulated by randomly retrieving pixels based on a perception-driven rule from an image to form a vector representing non-semantic global context, while a local Glance vector is a flattened local image window, mimicking a zoom-in observation. The Glance Index is defined as the inner product of two standardized sets of Glance vectors. We evaluate the effectiveness of incorporating Glance supervision in two reconstruction tasks: image fitting with implicit neural representation (INR) and undersampled MRI reconstruction. Extensive experimental results show that MS-Glance outperforms existing image restoration losses across both natural and medical images. The code is available at \url{https://github.com/Z7Gao/MSGlance}.