Application of Natural Language Processing to Determine User Satisfaction in Public Services

TL;DR

This paper explores how natural language processing, specifically topic models, can analyze large-scale open-ended user feedback to identify key factors influencing satisfaction in public healthcare services.

Contribution

It introduces a scalable NLP-based methodology using topic models to analyze open-ended feedback for public service improvement.

Findings

Staff interaction quality significantly impacts satisfaction.

Bureaucratic issues are a major dissatisfaction factor.

Analysis of 145,000 reviews reveals key satisfaction drivers.

Abstract

Research on customer satisfaction has increased substantially in recent years. However, the relative importance and relationships between different determinants of satisfaction remains uncertain. Moreover, quantitative studies to date tend to test for significance of pre-determined factors thought to have an influence with no scalable means to identify other causes of user satisfaction. The gaps in knowledge make it difficult to use available knowledge on user preference for public service improvement. Meanwhile, digital technology development has enabled new methods to collect user feedback, for example through online forums where users can comment freely on their experience. New tools are needed to analyze large volumes of such feedback. Use of topic models is proposed as a feasible solution to aggregate open-ended user opinions that can be easily deployed in the public sector. Generated insights can contribute to a more inclusive decision-making process in public service provision. This novel methodological approach is applied to a case of service reviews of publicly-funded primary care practices in England. Findings from the analysis of 145,000 reviews covering almost 7,700 primary care centers indicate that the quality of interactions with staff and bureaucratic exigencies are the key issues driving user satisfaction across England.