Analysis of the User Perception of Chatbots in Education Using A Partial Least Squares Structural Equation Modeling Approach

TL;DR

This study investigates factors influencing students' acceptance of educational chatbots, highlighting the roles of optimism, innovativeness, discomfort, and insecurity through PLS-SEM analysis of survey data.

Contribution

It applies PLS-SEM to explore key behavioral determinants of chatbot adoption in education, addressing gaps in understanding user perceptions and acceptance.

Findings

Optimism and innovativeness increase perceived ease of use and usefulness.

Discomfort and insecurity decrease perceived ease of use, with insecurity also reducing perceived usefulness.

Results inform chatbot design to enhance user acceptance in educational settings.

Abstract

The integration of Artificial Intelligence (AI) into education is a recent development, with chatbots emerging as a noteworthy addition to this transformative landscape. As online learning platforms rapidly advance, students need to adapt swiftly to excel in this dynamic environment. Consequently, understanding the acceptance of chatbots, particularly those employing Large Language Model (LLM) such as Chat Generative Pretrained Transformer (ChatGPT), Google Bard, and other interactive AI technologies, is of paramount importance. However, existing research on chatbots in education has overlooked key behavior-related aspects, such as Optimism, Innovativeness, Discomfort, Insecurity, Transparency, Ethics, Interaction, Engagement, and Accuracy, creating a significant literature gap. To address this gap, this study employs Partial Least Squares Structural Equation Modeling (PLS-SEM) to investigate the determinant of chatbots adoption in education among students, considering the Technology Readiness Index (TRI) and Technology Acceptance Model (TAM). Utilizing a five-point Likert scale for data collection, we gathered a total of 185 responses, which were analyzed using R-Studio software. We established 12 hypotheses to achieve its objectives. The results showed that Optimism and Innovativeness are positively associated with Perceived Ease of Use (PEOU) and Perceived Usefulness (PU). Conversely, Discomfort and Insecurity negatively impact PEOU, with only Insecurity negatively affecting PU. These findings provide insights for future technology designers, elucidating critical user behavior factors influencing chatbots adoption and utilization in educational contexts.