Decoding Reading Goals from Eye Movements
Omer Shubi, Cfir Avraham Hadar, Yevgeni Berzak
TL;DR
This study demonstrates that eye movements can be used to accurately decode whether a reader's goal is information seeking or comprehension, using advanced transformer models and real-time analysis.
Contribution
It introduces a novel approach combining transformer models, scanpath representations, and mixed effect modeling to decode reading goals from eye movements.
Findings
Transformer models with scanpath data outperform others.
Real-time prediction of reading goals is feasible.
Textual and participant factors influence decoding difficulty.
Abstract
Readers can have different goals with respect to the text that they are reading. Can these goals be decoded from their eye movements over the text? In this work, we examine for the first time whether it is possible to distinguish between two types of common reading goals: information seeking and ordinary reading for comprehension. Using large-scale eye tracking data, we address this task with a wide range of models that cover different architectural and data representation strategies, and further introduce a new model ensemble. We find that transformer-based models with scanpath representations coupled with language modeling solve it most successfully, and that accurate predictions can be made in real time, long before the participant finished reading the text. We further introduce a new method for model performance analysis based on mixed effect modeling. Combining this method with…
Peer Reviews
Decision·Submitted to ICLR 2025
The paper was a pleasure to read. It introduces all relevant background regarding eye movements and reading, has a clear outline and follows a nice story. Related work is exhaustive and paints a good picture of both machine learning models used in any kind of reading setting as well as eye-tracking-while-reading in general. They then introduce several machine learning models which consume either eye movements, the text that was read during the recording, or both.
The only true weakness is that the authors do not introduce a new model which exploits the eye movement while reading setting the authors investigate. Standard error not report, additionally, no statistical significance tests were done between trainable models, e.g. best model vs rest. The critical span (8./9. in the linear mixed effect model) for interpretability only works for already known texts. For binary classification AUROC would also be a great metric to report. Unfortunately, the an
Originality: The study introduces a novel problem - decoding reading goals from eye movements - that has not been widely explored. This new application area could encourage further research in cognitive science and multimodal data analysis. Quality: The use of diverse models and a comprehensive evaluation protocol ensures the quality and reliability of the findings. The error analysis is particularly valuable, as it provides insights into the factors that influence classification success, such a
Complexity in Model Descriptions: Some model descriptions lack clarity, especially in multimodal integration approaches. Providing visual diagrams or more intuitive breakdowns could make these sections more understandable. Limited Scope of Goal Types: The study only explores two reading goals (information seeking and ordinary reading). Extending this research to other goals (e.g., skimming, proofreading) could make the findings more broadly applicable.
Originality. The paper proposes an interesting research question, i.e. if one can distinguish between two reading tasks based on eye movements (+ text). This can be seen as an extension of previous work, in particular [1]. The authors suggest that their paper has broader scope and ecological validity than [1]. While this represents an important and valid extension, it is not particularly stark in originality. The methodologies are all adapted from previous works, as well as the data. While takin
- As discussed above, the paper is an interesting extension of previous work but does not particularly shine in terms of originality. - The authors do not discuss limitations of current methods or dataset. The authors briefly mention the need for different tasks and datasets in Section 8, but I believe a more structured and systematic discussion of limitations is necessary. For example: - The paragraphs considered in the study are all short. Do the findings generalise to longer text? - Bei
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsReading and Literacy Development