Comuniqa : Exploring Large Language Models for improving speaking skills

TL;DR

This paper introduces Comuniqa, an LLM-based system designed to improve English speaking skills, comparing its effectiveness with human expert feedback and highlighting its potential to enhance accessible language learning.

Contribution

The paper presents a novel LLM-based system, Comuniqa, for language learning, and evaluates its effectiveness against human expert guidance in improving speaking skills.

Findings

LLM-based system shows commendable accuracy in feedback

Comuniqa lacks human-level empathy and cognitive capabilities

System offers scalable and accessible language learning support

Abstract

In this paper, we investigate the potential of Large Language Models (LLMs) to improve English speaking skills. This is particularly relevant in countries like India, where English is crucial for academic, professional, and personal communication but remains a non-native language for many. Traditional methods for enhancing speaking skills often rely on human experts, which can be limited in terms of scalability, accessibility, and affordability. Recent advancements in Artificial Intelligence (AI) offer promising solutions to overcome these limitations. We propose Comuniqa, a novel LLM-based system designed to enhance English speaking skills. We adopt a human-centric evaluation approach, comparing Comuniqa with the feedback and instructions provided by human experts. In our evaluation, we divide the participants in three groups: those who use LLM-based system for improving speaking skills, those guided by human experts for the same task and those who utilize both the LLM-based system as well as the human experts. Using surveys, interviews, and actual study sessions, we provide a detailed perspective on the effectiveness of different learning modalities. Our preliminary findings suggest that while LLM-based systems have commendable accuracy, they lack human-level cognitive capabilities, both in terms of accuracy and empathy. Nevertheless, Comuniqa represents a significant step towards achieving Sustainable Development Goal 4: Quality Education by providing a valuable learning tool for individuals who may not have access to human experts for improving their speaking skills.