CLSP: High-Fidelity Contrastive Language-State Pre-training for Agent State Representation

TL;DR

This paper introduces CLSP, a contrastive pre-training method that significantly improves the accuracy and generalization of state representations in multimodal and reinforcement learning contexts, especially for numerical data.

Contribution

The paper presents a novel contrastive pre-training framework for high-fidelity state encoding, incorporating a classification task and RFF for numerical data enhancement, advancing state representation quality.

Findings

Superior performance in text-state retrieval tasks

Enhanced navigation capabilities in reinforcement learning

Improved understanding in multimodal large language models

Abstract

With the rapid development of artificial intelligence, multimodal learning has become an important research area. For intelligent agents, the state is a crucial modality to convey precise information alongside common modalities like images, videos, and language. This becomes especially clear with the broad adoption of reinforcement learning and multimodal large language models. Nevertheless, the representation of state modality still lags in development. To this end, we propose a High-Fidelity Contrastive Language-State Pre-training (CLSP) method, which can accurately encode state information into general representations for both reinforcement learning and multimodal large language models. Specifically, we first design a pre-training task based on the classification to train an encoder with coarse-grained information. Next, we construct data pairs of states and language descriptions, utilizing the pre-trained encoder to initialize the CLSP encoder. Then, we deploy contrastive learning to train the CLSP encoder to effectively represent precise state information. Additionally, we enhance the representation of numerical information using the Random Fourier Features (RFF) method for high-fidelity mapping. Extensive experiments demonstrate the superior precision and generalization capabilities of our representation, achieving outstanding results in text-state retrieval, reinforcement learning navigation tasks, and multimodal large language model understanding.