Learning Transferable Skills in Action RPGs via Directed Skill Graphs and Selective Adaptation

TL;DR

This paper introduces a hierarchical skill graph approach for lifelong learning in complex real-time environments, enabling efficient skill reuse and targeted adaptation to environmental changes in Dark Souls III.

Contribution

It proposes a skill graph framework with selective fine-tuning, improving sample efficiency and facilitating continual learning without retraining from scratch.

Findings

Targeted fine-tuning of two skills recovers performance rapidly.

Skill decomposition reduces training sample requirements.

Selective adaptation enables efficient environment shifts handling.

Abstract

Lifelong agents should expand their competence over time without retraining from scratch or overwriting previously learned behaviors. We investigate this in a challenging real-time control setting (Dark Souls III) by representing combat as a directed skill graph and training its components in a hierarchical curriculum. The resulting agent decomposes control into five reusable skills: camera control, target lock-on, movement, dodging, and a heal-attack decision policy, each optimized for a narrow responsibility. This factorization improves sample efficiency by reducing the burden on any single policy and supports selective post-training: when the environment shifts from Phase 1 to Phase 2, only a subset of skills must be adapted, while upstream skills remain transferable. Empirically, we find that targeted fine-tuning of just two skills rapidly recovers performance under a limited interaction budget, suggesting that skill-graph curricula together with selective fine-tuning offer a practical pathway toward evolving, continually learning agents in complex real-time environments.