See Your Heart: Psychological states Interpretation through Visual Creations
Likun Yang, Xiaokun Feng, Xiaotang Chen, Shiyu Zhang, Kaiqi Huang

TL;DR
This paper introduces VEIT, a new challenging task for AI to interpret psychological states from visual art, supported by a novel dataset and a model achieving state-of-the-art results, advancing psychoanalytic AI applications.
Contribution
The paper presents VEIT, a novel task for psychological interpretation from visual creations, along with the SpyIn dataset and a visual-semantic model that outperforms existing methods.
Findings
SpyIn dataset supports VEIT and is more challenging than existing captioning datasets.
The proposed model achieves state-of-the-art results on SpyIn.
VEIT requires scene graph and psychological knowledge for accurate interpretation.
Abstract
In psychoanalysis, generating interpretations to one's psychological state through visual creations is facing significant demands. The two main tasks of existing studies in the field of computer vision, sentiment/emotion classification and affective captioning, can hardly satisfy the requirement of psychological interpreting. To meet the demands for psychoanalysis, we introduce a challenging task, \textbf{V}isual \textbf{E}motion \textbf{I}nterpretation \textbf{T}ask (VEIT). VEIT requires AI to generate reasonable interpretations of creator's psychological state through visual creations. To support the task, we present a multimodal dataset termed SpyIn (\textbf{S}and\textbf{p}la\textbf{y} \textbf{In}terpretation Dataset), which is psychological theory supported and professional annotated. Dataset analysis illustrates that SpyIn is not only able to support VEIT, but also more challenging…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMultimodal Machine Learning Applications · Language, Metaphor, and Cognition · Topic Modeling
