Frog Soup: Zero-Shot, In-Context, and Sample-Efficient Frogger Agents

TL;DR

This paper explores how large language models can be used to play Frogger zero-shot, improve with in-context learning, and enhance traditional reinforcement learning methods, leading to more sample-efficient agents.

Contribution

It demonstrates zero-shot Frogger gameplay with LLMs, investigates in-context learning effects, and introduces a method to bootstrap RL with LLM demonstrations for better performance.

Findings

LLMs can play Frogger zero-shot after RL post-training.

In-context learning improves LLM performance on Frogger.

Bootstrapping RL with LLM demonstrations enhances sample efficiency.

Abstract

One of the primary aspirations in reinforcement learning research is developing general-purpose agents capable of rapidly adapting to and mastering novel tasks. While RL gaming agents have mastered many Atari games, they remain slow and costly to train for each game. In this work, we demonstrate that latest reasoning LLMs with out-of-domain RL post-training can play a challenging Atari game called Frogger under a zero-shot setting. We then investigate the effect of in-context learning and the amount of reasoning effort on LLM performance. Lastly, we demonstrate a way to bootstrap traditional RL method with LLM demonstrations, which significantly improves their performance and sample efficiency. Our implementation is open sourced at https://github.com/AlienKevin/frogger.