Validation of the Critical Reflection and Agency in Computing Index: Do Computing Ethics Courses Make a Difference?

TL;DR

This study validates an index measuring critical reflection and agency in computing ethics education, showing that ethics courses influence ethical development but may also reinforce techno-solutionist beliefs.

Contribution

The paper introduces a validated instrument to measure ethical development in computing students, enabling systematic assessment of ethics education effectiveness.

Findings

Participants in ethics courses scored higher on ethical reflection and agency.

Ethics courses increased critical consciousness but also strengthened techno-solutionist beliefs.

The index demonstrated strong reliability and construct validity.

Abstract

Computing ethics education aims to develop students' critical reflection and agency. We need validated ways to measure whether our efforts succeed. Through two survey administrations (N=474, N=464) with computing students and professionals, we provide evidence for the validity of the Critical Reflection and Agency in Computing Index. Our psychometric analyses demonstrate distinct dimensions of ethical development and show strong reliability and construct validity. Participants who completed computing ethics courses showed higher scores in some dimensions of ethical reflection and agency, but they also exhibited stronger techno-solutionist beliefs, highlighting a challenge in current pedagogy. This validated instrument enables systematic measurement of how computing students develop critical consciousness, allowing educators to better understand how to prepare computing professionals to tackle ethical challenges in their work.