SPGen: Stochastic scanpath generation for paintings using unsupervised domain adaptation
Mohamed Amine Kerkouri, Marouane Tliba, Aladine Chetouani, Alessandro Bruno

TL;DR
SPGen is a deep learning model that predicts human eye movement scanpaths on paintings by using unsupervised domain adaptation and stochastic modeling, aiding cultural heritage analysis.
Contribution
The paper introduces SPGen, a novel deep learning approach combining FCNN, domain adaptation, and stochastic sampling to predict scanpaths on artworks, addressing the domain gap from natural scenes.
Findings
SPGen outperforms existing gaze prediction methods
Effective transfer of knowledge from natural images to paintings
Models inherent stochasticity of eye movements
Abstract
Understanding human visual attention is key to preserving cultural heritage We introduce SPGen a novel deep learning model to predict scanpaths the sequence of eye movementswhen viewers observe paintings. Our architecture uses a Fully Convolutional Neural Network FCNN with differentiable fixation selection and learnable Gaussian priors to simulate natural viewing biases To address the domain gap between photographs and artworks we employ unsupervised domain adaptation via a gradient reversal layer allowing the model to transfer knowledge from natural scenes to paintings Furthermore a random noise sampler models the inherent stochasticity of eyetracking data. Extensive testing shows SPGen outperforms existing methods offering a powerful tool to analyze gaze behavior and advance the preservation and appreciation of artistic treasures.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsVisual Attention and Saliency Detection · Aesthetic Perception and Analysis · Gaze Tracking and Assistive Technology
