Questionnaires for Everyone: Streamlining Cross-Cultural Questionnaire Adaptation with GPT-Based Translation Quality Evaluation

TL;DR

This paper introduces a prototype tool that uses GPT-4 to evaluate and improve questionnaire translations, aiming to make cross-cultural research more accessible and less resource-intensive.

Contribution

The work presents a novel AI-assisted translation quality evaluation method that streamlines questionnaire adaptation across languages, reducing reliance on multiple human translators.

Findings

AI-assisted translations achieved comparable quality to traditional methods.

Participants could independently improve translations using GPT-4 suggestions.

The tool promotes more equitable and efficient cross-cultural questionnaire research.

Abstract

Adapting questionnaires to new languages is a resource-intensive process often requiring the hiring of multiple independent translators, which limits the ability of researchers to conduct cross-cultural research and effectively creates inequalities in research and society. This work presents a prototype tool that can expedite the questionnaire translation process. The tool incorporates forward-backward translation using DeepL alongside GPT-4-generated translation quality evaluations and improvement suggestions. We conducted two online studies in which participants translated questionnaires from English to either German (Study 1; n=10) or Portuguese (Study 2; n=20) using our prototype. To evaluate the quality of the translations created using the tool, evaluation scores between conventionally translated and tool-supported versions were compared. Our results indicate that integrating LLM-generated translation quality evaluations and suggestions for improvement can help users independently attain results similar to those provided by conventional, non-NLP-supported translation methods. This is the first step towards more equitable questionnaire-based research, powered by AI.