Survey Insights on M365 Copilot Adoption

TL;DR

This study evaluates the adoption of M365 Copilot in a scientific organization, revealing benefits in productivity and efficiency but also highlighting gaps, user satisfaction variability, and ethical concerns, informing future improvements.

Contribution

It provides empirical insights into M365 Copilot's real-world use in a research context, emphasizing user experiences, benefits, and challenges post-adoption.

Findings

Improved productivity in structured tasks like summaries and email drafting.

Identified gaps in advanced functionalities and integration.

Ethical concerns around data privacy increased after the trial.

Abstract

Australia's National Science Agency conducted a six-month trial of M365 Copilot starting in January 2024 as part of an Australian Government initiative. Three hundred licenses were distributed across CSIRO using a persona-based approach to ensure diversity of roles and attributes among participants. As a scientific research organisation with a unique operational context, our objective was to study the use of M365 Copilot on enhancing productivity, efficiency, and creativity as well as overall satisfaction and ethical considerations. This paper presents the results from two surveys, conducted before and after the trial. The results showed mixed outcomes, with participants reporting improved productivity and efficiency in structured tasks such as meeting summaries and email drafting. Gaps were identified in advanced functionalities and integration, which limited Copilot's overall effectiveness. Satisfaction levels were generally positive, but user experiences varied in certain areas. Ethical concerns, particularly around data privacy, became more pronounced post-trial. These findings highlight Copilot's potential and the need for further refinement to meet users' diverse needs.