Survey: Understand the challenges of MachineLearning Experts using Named EntityRecognition Tools

TL;DR

This survey explores the criteria, challenges, and decision-making processes of ML experts when selecting Named Entity Recognition tools, emphasizing their importance in clinical information retrieval and guideline development.

Contribution

It provides a comprehensive analysis of expert criteria and challenges in choosing NER tools, filling a gap in understanding the practical decision-making process in this domain.

Findings

Identification of key criteria used by ML experts for NER tool selection

Highlighting main challenges faced during NER tool evaluation

Insights into the decision-making process for NER tool adoption

Abstract

This paper presents a survey based on Kasunic's survey research methodology to identify the criteria used by Machine Learning (ML) experts to evaluate Named Entity Recognition (NER) tools and frameworks. Comparison and selection of NER tools and frameworks is a critical step in leveraging NER for Information Retrieval to support the development of Clinical Practice Guidelines. In addition, this study examines the main challenges faced by ML experts when choosing suitable NER tools and frameworks. Using Nunamaker's methodology, the article begins with an introduction to the topic, contextualizes the research, reviews the state-of-the-art in science and technology, and identifies challenges for an expert survey on NER tools and frameworks. This is followed by a description of the survey's design and implementation. The paper concludes with an evaluation of the survey results and the insights gained, ending with a summary and conclusions.