Just Noticeable Difference Modeling for Deep Visual Features

TL;DR

This paper introduces FeatJND, a task-aligned model for predicting the maximum imperceptible perturbation in deep visual features, improving feature quality control and downstream task performance.

Contribution

It proposes FeatJND, a novel task-aligned JND formulation for deep features, with a predictor that enhances feature quality management across various vision tasks.

Findings

FeatJND preserves higher task performance under distortions.

It suppresses non-critical feature regions effectively.

Guides dynamic quantization for better resource utilization.

Abstract

Deep visual features are increasingly used as the interface in vision systems, motivating the need to describe feature characteristics and control feature quality for machine perception. Just noticeable difference (JND) characterizes the maximum imperceptible distortion for images under human or machine vision. Extending it to deep visual features naturally meets the above demand by providing a task-aligned tolerance boundary in feature space, offering a practical reference for controlling feature quality under constrained resources. We propose FeatJND, a task-aligned JND formulation that predicts the maximum tolerable per-feature perturbation map while preserving downstream task performance. We propose a FeatJND estimator at standardized split points and validate it across image classification, detection, and instance segmentation. Under matched distortion strength, FeatJND-based distortions consistently preserve higher task performance than unstructured Gaussian perturbations, and attribution visualizations suggest FeatJND can suppress non-critical feature regions. As an application, we further apply FeatJND to token-wise dynamic quantization and show that FeatJND-guided step-size allocation yields clear gains over random step-size permutation and global uniform step size under the same noise budget. Our code will be released after publication.