SPGen: Stochastic scanpath generation for paintings using unsupervised domain adaptation

TL;DR

SPGen is a deep learning model that predicts human eye movement scanpaths on paintings by using unsupervised domain adaptation and stochastic modeling, aiding cultural heritage analysis.

Contribution

The paper introduces SPGen, a novel deep learning approach combining FCNN, domain adaptation, and stochastic sampling to predict scanpaths on artworks, addressing the domain gap from natural scenes.

Findings

SPGen outperforms existing gaze prediction methods

Effective transfer of knowledge from natural images to paintings

Models inherent stochasticity of eye movements

Abstract

Understanding human visual attention is key to preserving cultural heritage We introduce SPGen a novel deep learning model to predict scanpaths the sequence of eye movementswhen viewers observe paintings. Our architecture uses a Fully Convolutional Neural Network FCNN with differentiable fixation selection and learnable Gaussian priors to simulate natural viewing biases To address the domain gap between photographs and artworks we employ unsupervised domain adaptation via a gradient reversal layer allowing the model to transfer knowledge from natural scenes to paintings Furthermore a random noise sampler models the inherent stochasticity of eyetracking data. Extensive testing shows SPGen outperforms existing methods offering a powerful tool to analyze gaze behavior and advance the preservation and appreciation of artistic treasures.