An Exploration-Analysis-Disambiguation Reasoning Framework for Word Sense Disambiguation with Low-Parameter LLMs

TL;DR

This paper shows that small-scale, low-parameter LLMs can perform effective Word Sense Disambiguation comparable to large models like GPT-4-Turbo by using reasoning-focused fine-tuning and Chain-of-Thought prompting, reducing computational costs.

Contribution

It introduces a reasoning-driven fine-tuning approach for low-parameter LLMs that achieves competitive WSD performance and strong cross-domain generalization.

Findings

Low-parameter LLMs with reasoning strategies match GPT-4-Turbo in zero-shot WSD.

Gemma-3-4B and Qwen-3-4B outperform larger baselines on FEWS.

Models generalize well to unseen senses and domains without task-specific fine-tuning.

Abstract

Word Sense Disambiguation (WSD) remains a key challenge in Natural Language Processing (NLP), especially when dealing with rare or domain-specific senses that are often misinterpreted. While modern high-parameter Large Language Models (LLMs) such as GPT-4-Turbo have shown state-of-the-art WSD performance, their computational and energy demands limit scalability. This study investigates whether low-parameter LLMs (<4B parameters) can achieve comparable results through fine-tuning strategies that emphasize reasoning-driven sense identification. Using the FEWS dataset augmented with semi-automated, rationale-rich annotations, we fine-tune eight small-scale open-source LLMs (e.g. Gemma and Qwen). Our results reveal that Chain-of-Thought (CoT)-based reasoning combined with neighbour-word analysis achieves performance comparable to GPT-4-Turbo in zero-shot settings. Importantly, Gemma-3-4B and Qwen-3-4B models consistently outperform all medium-parameter baselines and state-of-the-art models on FEWS, with robust generalization to unseen senses. Furthermore, evaluation on the unseen "Fool Me If You Can'' dataset confirms strong cross-domain adaptability without task-specific fine-tuning. This work demonstrates that with carefully crafted reasoning-centric fine-tuning, low-parameter LLMs can deliver accurate WSD while substantially reducing computational and energy demands.