JAM: Controllable and Responsible Text Generation via Causal Reasoning and Latent Vector Manipulation

TL;DR

JAM introduces a causal reasoning framework that interprets and controls large language model outputs by manipulating latent vectors, significantly improving responsible and realistic text generation.

Contribution

This paper presents JAM, a novel causal reasoning approach that enhances interpretability and control in LLMs through latent vector manipulation, with improved efficiency and performance.

Findings

Up to 22% improvement over previous methods in quantitative metrics

Demonstrates greater computational efficiency

Achieves responsible and realistic text generation

Abstract

While large language models (LLMs) have made significant strides in generating coherent and contextually relevant text, they often function as opaque black boxes, trained on vast unlabeled datasets with statistical objectives, lacking an interpretable framework for responsible control. In this paper, we introduce JAM (Just A Move), a novel framework that interprets and controls text generation by integrating cause-effect analysis within the latent space of LLMs. Based on our observations, we uncover the inherent causality in LLM generation, which is critical for producing responsible and realistic outputs. Moreover, we explore latent vectors as fundamental components in LLM architectures, aiming to understand and manipulate them for more effective and efficient controllable text generation. We evaluate our framework using a range of tools, including the HHH criteria, toxicity reduction benchmarks, and GPT-4 alignment measures. Our results show that JAM achieves up to a 22% improvement over previous Controllable Text Generation (CTG) methods across multiple quantitative metrics and human-centric evaluations. Furthermore, JAM demonstrates greater computational efficiency compared to other CTG methods. These results highlight the effectiveness and efficiency of JAM for responsible and realistic text generation, paving the way for more interpretable and controllable models.