How much reliable is ChatGPT's prediction on Information Extraction under Input Perturbations?

TL;DR

This paper systematically evaluates ChatGPT's robustness in Named Entity Recognition under input perturbations, revealing its brittleness, overconfidence, and the impact of perturbations on explanation quality, with implications for its reliability in real-world applications.

Contribution

It provides a comprehensive analysis of ChatGPT's robustness in NER tasks under various input perturbations, highlighting its limitations and the effects on prediction confidence and explanation quality.

Findings

ChatGPT is more brittle on rare entity perturbations like drugs or diseases.

Explanation quality varies significantly with different perturbations.

ChatGPT tends to be overconfident in incorrect predictions.

Abstract

In this paper, we assess the robustness (reliability) of ChatGPT under input perturbations for one of the most fundamental tasks of Information Extraction (IE) i.e. Named Entity Recognition (NER). Despite the hype, the majority of the researchers have vouched for its language understanding and generation capabilities; a little attention has been paid to understand its robustness: How the input-perturbations affect 1) the predictions, 2) the confidence of predictions and 3) the quality of rationale behind its prediction. We perform a systematic analysis of ChatGPT's robustness (under both zero-shot and few-shot setup) on two NER datasets using both automatic and human evaluation. Based on automatic evaluation metrics, we find that 1) ChatGPT is more brittle on Drug or Disease replacements (rare entities) compared to the perturbations on widely known Person or Location entities, 2) the quality of explanations for the same entity considerably differ under different types of "Entity-Specific" and "Context-Specific" perturbations and the quality can be significantly improved using in-context learning, and 3) it is overconfident for majority of the incorrect predictions, and hence it could lead to misguidance of the end-users.