VS-LLM: Visual-Semantic Depression Assessment based on LLM for Drawing Projection Test

TL;DR

This paper introduces VS-LLM, a novel method leveraging large language models to automatically assess depression from Drawing Projection Test sketches, enhancing accuracy and efficiency in mental health evaluation.

Contribution

The paper presents a new automated depression assessment approach based on visual-semantic analysis of PPAT sketches using LLMs, addressing interpretation challenges in art therapy.

Findings

Improved depression detection accuracy by 17.6% over traditional methods.

Developed an environment for automated PPAT sketch analysis.

Contributed datasets and code for future research.

Abstract

The Drawing Projection Test (DPT) is an essential tool in art therapy, allowing psychologists to assess participants' mental states through their sketches. Specifically, through sketches with the theme of "a person picking an apple from a tree (PPAT)", it can be revealed whether the participants are in mental states such as depression. Compared with scales, the DPT can enrich psychologists' understanding of an individual's mental state. However, the interpretation of the PPAT is laborious and depends on the experience of the psychologists. To address this issue, we propose an effective identification method to support psychologists in conducting a large-scale automatic DPT. Unlike traditional sketch recognition, DPT more focus on the overall evaluation of the sketches, such as color usage and space utilization. Moreover, PPAT imposes a time limit and prohibits verbal reminders, resulting in low drawing accuracy and a lack of detailed depiction. To address these challenges, we propose the following efforts: (1) Providing an experimental environment for automated analysis of PPAT sketches for depression assessment; (2) Offering a Visual-Semantic depression assessment based on LLM (VS-LLM) method; (3) Experimental results demonstrate that our method improves by 17.6% compared to the psychologist assessment method. We anticipate that this work will contribute to the research in mental state assessment based on PPAT sketches' elements recognition. Our datasets and codes are available at https://github.com/wmeiqi/VS-LLM.