EMBRACE: Shaping Inclusive Opinion Representation by Aligning Implicit Conversations with Social Norms

TL;DR

This paper introduces EMBRACE, a framework for evaluating and improving inclusive opinion representation in NLP models by focusing on implicit conversations and social norms, using stance modeling and alignment assessment.

Contribution

It proposes a novel alignment evaluation framework that emphasizes implicit opinions and normative social views, advancing beyond surface-level demographic inclusion methods.

Findings

The framework effectively models implicit opinions through stance analysis.

Evaluation with online learning and instruction-tuned models reveals alignment issues.

The approach offers a structured way to assess and enhance social norm alignment in NLP models.

Abstract

Shaping inclusive representations that embrace diversity and ensure fair participation and reflections of values is at the core of many conversation-based models. However, many existing methods rely on surface inclusion using mention of user demographics or behavioral attributes of social groups. Such methods overlook the nuanced, implicit expression of opinion embedded in conversations. Furthermore, the over-reliance on overt cues can exacerbate misalignment and reinforce harmful or stereotypical representations in model outputs. Thus, we took a step back and recognized that equitable inclusion needs to account for the implicit expression of opinion and use the stance of responses to validate the normative alignment. This study aims to evaluate how opinions are represented in NLP or computational models by introducing an alignment evaluation framework that foregrounds implicit, often overlooked conversations and evaluates the normative social views and discourse. Our approach models the stance of responses as a proxy for the underlying opinion, enabling a considerate and reflective representation of diverse social viewpoints. We evaluate the framework using both (i) positive-unlabeled (PU) online learning with base classifiers, and (ii) instruction-tuned language models to assess post-training alignment. Through this, we provide a principled and structured lens on how implicit opinions are (mis)represented and offer a pathway toward more inclusive model behavior.