Recognizing Instagram Filtered Images with Feature De-stylization
Zhe Wu, Zuxuan Wu, Bharat Singh, Larry S. Davis

TL;DR
This paper investigates how Instagram filters affect pretrained neural networks and introduces a lightweight de-stylization module that improves model robustness against such filters by normalizing feature space alterations.
Contribution
The paper presents a novel lightweight de-stylization module that can be integrated into CNNs to counteract the effects of Instagram filters, enhancing robustness without full retraining.
Findings
Filters cause significant feature space shifts in CNNs.
The de-stylization module improves generalization on filtered images.
Normalization parameters can be learned without retraining the entire network.
Abstract
Deep neural networks have been shown to suffer from poor generalization when small perturbations are added (like Gaussian noise), yet little work has been done to evaluate their robustness to more natural image transformations like photo filters. This paper presents a study on how popular pretrained models are affected by commonly used Instagram filters. To this end, we introduce ImageNet-Instagram, a filtered version of ImageNet, where 20 popular Instagram filters are applied to each image in ImageNet. Our analysis suggests that simple structure preserving filters which only alter the global appearance of an image can lead to large differences in the convolutional feature space. To improve generalization, we introduce a lightweight de-stylization module that predicts parameters used for scaling and shifting feature maps to "undo" the changes incurred by filters, inverting the process…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGenerative Adversarial Networks and Image Synthesis · Human Pose and Action Recognition · Face recognition and analysis
