Fine-tuning vs Prompting, Can Language Models Understand Human Values?

TL;DR

This paper investigates the effectiveness of fine-tuning and prompt tuning methods in enabling large language models to understand and detect human values in sentences, comparing their performance and exploring pre-training knowledge utilization.

Contribution

It provides a comparative analysis of fine-tuning and prompt tuning for human value detection and assesses the pre-training knowledge's role in this task.

Findings

Prompt tuning shows competitive performance to fine-tuning.

Pre-training knowledge contributes significantly to value detection.

Large language models with RLHF can partially understand human values.

Abstract

Accurately handling the underlying support values in sentences is crucial for understanding the speaker's tendencies, yet it poses a challenging task in natural language understanding (NLU). In this article, we explore the potential of fine-tuning and prompt tuning in this downstream task, using the Human Value Detection 2023. Additionally, we attempt to validate whether models can effectively solve the problem based on the knowledge acquired during the pre-training stage. Simultaneously, our interest lies in the capabilities of large language models (LLMs) aligned with RLHF in this task, and some preliminary attempts are presented.