Formal Semantic Control over Language Models

TL;DR

This thesis develops methods to improve the interpretability and controllability of language models' semantic representations by shaping their latent space within a VAE framework, enabling targeted manipulation at sentence and reasoning levels.

Contribution

It introduces novel theoretical frameworks and practical techniques for disentangling and controlling semantic features in language models' latent spaces within a VAE architecture.

Findings

Enhanced interpretability of semantic representations

Improved controllability of language model outputs

Effective manipulation of inference behaviors in NLI tasks

Abstract

This thesis advances semantic representation learning to render language representations or models more semantically and geometrically interpretable, and to enable localised, quasi-symbolic, compositional control through deliberate shaping of their latent space geometry. We pursue this goal within a VAE framework, exploring two complementary research directions: (i) Sentence-level learning and control: disentangling and manipulating specific semantic features in the latent space to guide sentence generation, with explanatory text serving as the testbed; and (ii) Reasoning-level learning and control: isolating and steering inference behaviours in the latent space to control NLI. In this direction, we focus on Explanatory NLI tasks, in which two premises (explanations) are provided to infer a conclusion. The overarching objective is to move toward language models whose internal semantic representations can be systematically interpreted, precisely structured, and reliably directed. We introduce a set of novel theoretical frameworks and practical methodologies, together with corresponding experiments, to demonstrate that our approaches enhance both the interpretability and controllability of latent spaces for natural language across the thesis.