Subjective Behaviors and Preferences in LLM: Language of Browsing

TL;DR

This paper introduces HeTLM, a clusterwise training method for LLMs that better captures subjective browsing behaviors, showing small models can outperform larger ones and that heterogeneity-aware training improves user-level alignment.

Contribution

The paper proposes HeTLM, a novel training approach for LLMs that accounts for user heterogeneity, demonstrating improved performance and alignment over traditional single-model methods.

Findings

Small LMs with page-level tokenizers outperform large pretrained LMs.

HeTLM with cluster-specific parameters outperforms single LMs of similar size.

HeTLM achieves higher mean performance and lower variance, enhancing user alignment.

Abstract

A Large Language Model (LLM) offers versatility across domains and tasks, purportedly benefiting users with a wide variety of behaviors and preferences. We question this perception about an LLM when users have inherently subjective behaviors and preferences, as seen in their ubiquitous and idiosyncratic browsing of websites or apps. The sequential behavior logs of pages, thus generated, form something akin to each user's self-constructed "language", albeit without the structure and grammar imbued in natural languages. We ask: (i) Can a small LM represent the "language of browsing" better than a large LM? (ii) Can an LM with a single set of parameters (or, single LM) adequately capture myriad users' heterogeneous, subjective behaviors and preferences? (iii) Can a single LM with high average performance, yield low variance in performance to make alignment good at user level? We introduce clusterwise LM training, HeTLM (Heterogeneity aware Training of Language Model), appropriate for subjective behaviors. We find that (i) a small LM trained using a page-level tokenizer outperforms large pretrained or finetuned LMs; (ii) HeTLM with heterogeneous cluster specific set of parameters outperforms a single LM of the same family, controlling for the number of parameters; and (iii) a higher mean and a lower variance in generation ensues, implying improved alignment.