Enhancement of Subjective Content Descriptions by using Human Feedback

TL;DR

This paper introduces ReFrESH, a method that uses human feedback to improve subjective content descriptions in text retrieval systems, making answers more aligned with individual user perceptions.

Contribution

It presents a novel approach for incrementally updating subjective content descriptions using human feedback, addressing ambiguity and preserving relations among descriptions.

Findings

ReFrESH effectively updates SCDs with human feedback.

The approach improves answer relevance to user perceptions.

It manages feedback ambiguity and relation preservation.

Abstract

An agent providing an information retrieval service may work with a corpus of text documents. The documents in the corpus may contain annotations such as Subjective Content Descriptions (SCD) -- additional data associated with different sentences of the documents. Each SCD is associated with multiple sentences of the corpus and has relations among each other. The agent uses the SCDs to create its answers in response to queries supplied by users. However, the SCD the agent uses might reflect the subjective perspective of another user. Hence, answers may be considered faulty by an agent's user, because the SCDs may not exactly match the perceptions of an agent's user. A naive and very costly approach would be to ask each user to completely create all the SCD themselves. To use existing knowledge, this paper presents ReFrESH, an approach for Relation-preserving Feedback-reliant Enhancement of SCDs by Humans. An agent's user can give feedback about faulty answers to the agent. This feedback is then used by ReFrESH to update the SCDs incrementally. However, human feedback is not always unambiguous. Therefore, this paper additionally presents an approach to decide how to incorporate the feedback and when to update the SCDs. Altogether, SCDs can be updated with human feedback, allowing users to create even more specific SCDs for their needs.