InterventionLens: A Multi-Agent Framework for Detecting ASD Intervention Strategies in Parent-Child Shared Reading

TL;DR

InterventionLens is an innovative multi-agent system that automatically detects and segments caregiver intervention strategies in shared reading videos for children with ASD, reducing reliance on expert annotation.

Contribution

It introduces a task-agnostic, multi-agent framework that integrates multimodal data for fine-grained analysis without model fine-tuning.

Findings

Achieves 79.44% F1 score in intervention detection

Outperforms baseline by 19.72%

Effective in naturalistic home settings

Abstract

Home-based interventions like parent-child shared reading provide a cost-effective approach for supporting children with autism spectrum disorder (ASD). However, analyzing caregiver intervention strategies in naturalistic home interactions typically relies on expert annotation, which is costly, time-intensive, and difficult to scale. To address this challenge, we propose InterventionLens, an end-to-end multi-agent system for automatically detecting and temporally segmenting caregiver intervention strategies from shared reading videos. Without task-specific model training or fine-tuning, InterventionLens uses a collaborative multi-agent architecture to integrate multimodal interaction content and perform fine-grained strategy analysis. Experiments on the ASD-HI dataset show that InterventionLens achieves an overall F1 score of 79.44\%, outperforming the baseline by 19.72\%. These results suggest that InterventionLens is a promising system for analyzing caregiver intervention strategies in home-based ASD shared reading settings. Additional resources will be released on the project page.