Semantic Distance Measurement based on Multi-Kernel Gaussian Processes

TL;DR

This paper introduces a novel semantic distance measure using multi-kernel Gaussian processes, which adaptively learns kernel parameters from data, improving tasks like sentiment classification with large language models.

Contribution

The paper proposes a flexible semantic distance method based on MK-GP that automatically learns kernel parameters, enhancing adaptability over classical fixed methods.

Findings

Effective in fine-grained sentiment classification

Outperforms classical semantic distance methods

Demonstrates robustness with large language models

Abstract

Semantic distance measurement is a fundamental problem in computational linguistics, providing a quantitative characterization of similarity or relatedness between text segments, and underpinning tasks such as text retrieval and text classification. From a mathematical perspective, a semantic distance can be viewed as a metric defined on a space of texts or on a representation space derived from them. However, most classical semantic distance methods are essentially fixed, making them difficult to adapt to specific data distributions and task requirements. In this paper, a semantic distance measure based on multi-kernel Gaussian processes (MK-GP) was proposed. The latent semantic function associated with texts was modeled as a Gaussian process, with its covariance function given by a combined kernel combining Mat\'ern and polynomial components. The kernel parameters were learned automatically from data under supervision, rather than being hand-crafted. This semantic distance was instantiated and evaluated in the context of fine-grained sentiment classification with large language models under an in-context learning (ICL) setup. The experimental results demonstrated the effectiveness of the proposed measure.