Campus AI vs Commercial AI: A Late-Breaking Study on How LLM As-A-Service Customizations Shape Trust and Usage Patterns

TL;DR

This study investigates how user-focused customizations of LLM-as-a-Service influence trust and usage, comparing institutional models to commercial options, to inform better deployment strategies.

Contribution

It highlights the importance of user-centric customizations like interface and branding in shaping trust and usage patterns of LLMaaS, a less-studied aspect.

Findings

Preliminary insights into trust differences between campus and commercial LLMs

Identification of user interface and branding as key trust factors

Framework for assessing psychological impacts of LLM customizations

Abstract

As the use of Large Language Models (LLMs) by students, lecturers and researchers becomes more prevalent, universities - like other organizations - are pressed to develop coherent AI strategies. LLMs as-a-Service (LLMaaS) offer accessible pre-trained models, customizable to specific (business) needs. While most studies prioritize data, model, or infrastructure adaptations (e.g., model fine-tuning), we focus on user-salient customizations, like interface changes and corporate branding, which we argue influence users' trust and usage patterns. This study serves as a functional prequel to a large-scale field study in which we examine how students and employees at a German university perceive and use their institution's customized LLMaaS compared to ChatGPT. The goals of this prequel are to stimulate discussions on psychological effects of LLMaaS customizations and refine our research approach through feedback. Our forthcoming findings will deepen the understanding of trust dynamics in LLMs, providing practical guidance for organizations considering LLMaaS deployment.