The use of GPT-4o and Other Large Language Models for the Improvement and Design of Self-Assessment Scales for Measurement of Interpersonal Communication Skills

TL;DR

This paper explores how GPT-4o and other large language models can enhance the development and refinement of self-assessment scales for measuring interpersonal communication skills, leveraging their advanced reasoning and content generation capabilities.

Contribution

It demonstrates the novel application of LLMs in designing, evaluating, and improving interpersonal communication assessment tools through case studies and specific prompts.

Findings

LLMs can assist in item selection and content validity evaluation.

Automated item generation is feasible with LLMs.

Using LLMs improves the efficiency of scale development processes.

Abstract

OpenAI's ChatGPT (GPT-4 and GPT-4o) and other Large Language Models (LLMs) like Microsoft's Copilot, Google's Gemini 1.5 Pro, and Antrophic's Claude 3.5 Sonnet can be effectively used in various phases of scientific research. Their performance in diverse verbal tasks and reasoning is close to or above the average human level and rapidly increasing, providing those models with a capacity that resembles a relatively high level of theory of mind. The current ability of LLMs to process information about human psychology and communication creates an opportunity for their scientific use in the fields of personality psychology and interpersonal communication skills. This article illustrates the possible uses of GPT-4o and other advanced LLMs for typical tasks in designing self-assessment scales for interpersonal communication skills measurement like the selection and improvement of scale items and evaluation of content validity of scales. The potential for automated item generation and application is illustrated as well. The case study examples are accompanied by prompts for LLMs that can be useful for these purposes. Finally, a summary is provided of the potential benefits of using LLMs in the process of evaluation, design, and improvement of interpersonal communication skills self-assessment scales.