Indexing Analytics to Instances: How Integrating a Dashboard can Support Design Education

TL;DR

This paper explores how integrating AI-based multiscale design analytics into a dashboard linked with actual student design work can enhance instructors' understanding and support in design education.

Contribution

It introduces a novel research artifact that connects design analytics with actual design instances, supporting instructor reflection and assessment in situated educational contexts.

Findings

Analytics linked to design instances improve instructor understanding.

Multiscale analytics support nuanced assessment and feedback.

The dashboard facilitates mutual intelligibility between AI insights and design work.

Abstract

We investigate how to use AI-based analytics to support design education. The analytics at hand measure multiscale design, that is, students' use of space and scale to visually and conceptually organize their design work. With the goal of making the analytics intelligible to instructors, we developed a research artifact integrating a design analytics dashboard with design instances, and the design environment that students use to create them. We theorize about how Suchman's notion of mutual intelligibility requires contextualized investigation of AI in order to develop findings about how analytics work for people. We studied the research artifact in 5 situated course contexts, in 3 departments. A total of 236 students used the multiscale design environment. The 9 instructors who taught those students experienced the analytics via the new research artifact. We derive findings from a qualitative analysis of interviews with instructors regarding their experiences. Instructors reflected on how the analytics and their presentation in the dashboard have the potential to affect design education. We develop research implications addressing: (1) how indexing design analytics in the dashboard to actual design work instances helps design instructors reflect on what they mean and, more broadly, is a technique for how AI-based design analytics can support instructors' assessment and feedback experiences in situated course contexts; and (2) how multiscale design analytics, in particular, have the potential to support design education. By indexing, we mean linking which provides context, here connecting the numbers of the analytics with visually annotated design work instances.