Solving NLP Problems through Human-System Collaboration: A Discussion-based Approach

TL;DR

This paper introduces a discussion-based framework enabling AI systems to engage in dialogues with humans to collaboratively refine predictions, significantly improving accuracy in natural language inference tasks.

Contribution

It presents a novel dataset and computational framework for systems to have mutual discussions with humans, advancing explainability and collaborative problem-solving in NLP.

Findings

System improves natural language inference accuracy by up to 25 points

Dialogue-based approach enhances system reliability and human understanding

Proposed framework facilitates mutual exchange of opinions in NLP tasks

Abstract

Humans work together to solve common problems by having discussions, explaining, and agreeing or disagreeing with each other. Similarly, if a system can have discussions with humans when solving tasks, it can improve the system's performance and reliability. In previous research on explainability, it has only been possible for the system to make predictions and for humans to ask questions about them rather than having a mutual exchange of opinions. This research aims to create a dataset and computational framework for systems that discuss and refine their predictions through dialogue. Through experiments, we show that the proposed system can have beneficial discussions with humans improving the accuracy by up to 25 points in the natural language inference task.