Interpretability by design using computer vision for behavioral sensing in child and adolescent psychiatry

TL;DR

This paper presents a computer vision approach to automatically code behavioral states in children and adolescents during clinical interviews, aiming to improve reliability and efficiency in mental health assessments.

Contribution

It introduces a computer vision-based method that aligns with standard behavioral rating systems, enhancing interpretability for mental health professionals.

Findings

ML ratings matched expert ratings for negative emotions, activity, and anxiety.

Reasonable performance for attention and positive affect concepts.

Gaze and vocalization results suggest need for better data or modalities.

Abstract

Observation is an essential tool for understanding and studying human behavior and mental states. However, coding human behavior is a time-consuming, expensive task, in which reliability can be difficult to achieve and bias is a risk. Machine learning (ML) methods offer ways to improve reliability, decrease cost, and scale up behavioral coding for application in clinical and research settings. Here, we use computer vision to derive behavioral codes or concepts of a gold standard behavioral rating system, offering familiar interpretation for mental health professionals. Features were extracted from videos of clinical diagnostic interviews of children and adolescents with and without obsessive-compulsive disorder. Our computationally-derived ratings were comparable to human expert ratings for negative emotions, activity-level/arousal and anxiety. For the attention and positive affect concepts, our ML ratings performed reasonably. However, results for gaze and vocalization indicate a need for improved data quality or additional data modalities.