LinguaFluid: Language Guided Fluid Control via Semantic Rewards in Reinforcement Learning

TL;DR

This paper presents LinguaFluid, a reinforcement learning approach that uses semantic similarity between language instructions and states to guide control tasks, reducing reliance on manual reward engineering.

Contribution

It introduces a novel semantic reward mechanism using SBERT for aligning language instructions with agent states in RL environments.

Findings

Semantic rewards guide learning effectively without hand-crafted rewards.

Language embedding space correlates with Euclidean space in control tasks.

Framework enables natural language goal alignment in fluid control.

Abstract

In the domain of scientific machine learning, designing effective reward functions remains a challenge in reinforcement learning (RL), particularly in environments where task goals are difficult to specify numerically. Reward functions in existing work are predominantly based on heuristics, manual engineering, or task-specific tuning. In this work, we introduce a semantically aligned reinforcement learning method where rewards are computed by aligning the current state with a target semantic instruction using a Sentence-Bidirectional Encoder Representations from Transformers (SBERT). Instead of relying on manually defined reward functions, the policy receives feedback based on the reward, which is a cosine similarity between the goal textual description and the statement description in the episode. We evaluated our approach in several environments and showed that semantic reward can guide learning to achieve competitive control behavior, even in the absence of hand-crafted reward functions. Our study demonstrates a correlation between the language embedding space and the conventional Euclidean space. This framework opens new horizons for aligning agent behavior with natural language goals and lays the groundwork for a more seamless integration of larger language models (LLMs) and fluid control applications.