Evaluating an Immersive Analytics Application at an Enterprise Business Intelligence Customer Conference

TL;DR

This study evaluates the usability and potential of an immersive analytics application, Tableau for visionOS, at a large enterprise BI conference, highlighting evaluation challenges and insights for future immersive analytics assessments.

Contribution

It provides a novel enterprise perspective on evaluation methodologies for immersive analytics, integrating qualitative and quantitative measures in a real-world setting.

Findings

Usability of Tableau for visionOS was assessed with 22 participants.

Participants provided insights on the practicality and novelty of immersive BI interfaces.

The study highlights the need for new evaluation methods for HMD-based analytics.

Abstract

We reflect on an evaluation of an immersive analytics application (Tableau for visionOS) conducted at a large enterprise business intelligence (BI) conference. Conducting a study in such a context offered an opportunistic setting to gather diverse feedback. However, this setting also highlighted the challenge of evaluating usability while also assessing potential utility, as feedback straddled between the novelty of the experience and the practicality of the application in participants' analytical workflows. This formative evaluation with 22 participants allowed us to gather insights with respect to the usability of Tableau for visionOS, along with broader perspectives on the potential for head-mounted displays (HMDs) to promote new ways to engage with BI data. Our experience suggests a need for new evaluation considerations that integrate qualitative and quantitative measures and account for unique interaction patterns with 3D representations and interfaces accessible via an HMD. Overall, we contribute an enterprise perspective on evaluation methodologies for immersive analytics.