Perceived Usability of Collaborative Modeling Tools

TL;DR

This study compares the perceived usability of chatbot-based collaborative modeling tools and traditional web-based tools, finding chatbots improve collaboration efficiency and user experience but need better guidance for novices.

Contribution

It provides empirical evidence on the usability of chatbot-based collaborative modeling tools versus traditional tools, highlighting benefits and areas for improvement.

Findings

Chatbots enhance communication and collaboration efficiency.

Chatbots improve overall user experience.

Guidance for novices in chatbots needs improvement.

Abstract

Context: Online collaborative creation of models is becoming commonplace. Collaborative modeling using chatbots and natural language may lower the barriers to modeling for users from different domains. Objective: We compare the perceived usability of two similarly online collaborative modeling tools, the SOCIO chatbot and the Creately web-based tool. Method: We conducted a crossover experiment with 66 participants. The evaluation instrument was based on the System Usability Scale (SUS). We performed a quantitative and qualitative exploration, employing inferential statistics and thematic analysis. Results: The results indicate that chatbots enabling natural language communication enhance communication and collaboration efficiency and improve the user experience. Conclusion: Chatbots need to improve guidance and help for novices, but they appear beneficial for enhancing user experience.