ConvScale: Conversational Interviews for Scale-Aligned Measurement
Peinuan Qin, Jingzhu Chen, Yitian Yang, Han Meng, Zicheng Zhu, Yi-Chieh Lee

TL;DR
ConvScale introduces an AI-supported method to transform psychometric scales into conversational interviews, aiming to enhance quantitative measurement by predicting item scores from natural dialogue, with promising alignment to self-reports.
Contribution
This paper presents ConvScale, a novel approach that converts psychometric scales into conversational interviews and predicts quantitative scores from dialogue data.
Findings
Scores predicted by ConvScale closely match self-reports at item and construct levels
Moderate internal reliability achieved in the conversational assessment
Structural validity of the conversational approach was found to be inadequate
Abstract
Conversational interviews are commonly used to complement structured surveys by eliciting rich and contextualized responses, which are typically analyzed qualitatively. However, their potential contribution to quantitative measurement remains underexplored. In this paper, we introduce ConvScale, an AI-supported approach that transforms psychometric scales into natural conversational interviews while preserving the original measurement structure. Based on interview data, ConvScale predicts item-level scores and aggregates them to derive scale-based assessments. In a within-subjects study with 18 participants, our results show that ConvScale-derived scores align closely with participants' self-report scores at both the item and construct levels, while maintaining moderate internal reliability; however, the structural validity was inadequate. In light of this, we discussed the potential of…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsPsychometric Methodologies and Testing · Survey Methodology and Nonresponse · Mental Health via Writing
