# Machine Teaching in Hierarchical Genetic Reinforcement Learning: Curriculum Design of Reward Functions for Swarm Shepherding

## TL;DR

This paper introduces a systematic instructional design approach to guide reward function creation in hierarchical genetic reinforcement learning, enabling more effective swarm control in agent shepherding tasks.

## Contribution

It presents a novel methodology for systematic reward function design in hierarchical reinforcement learning using human education principles.

## Key findings

- Methodology successfully guides reward function design for hierarchical learners.
- Hierarchical models learn incrementally through multi-part reward functions.
- Hierarchy effectively combines behaviors for smart swarm shepherding.

## Abstract

The design of reward functions in reinforcement learning is a human skill that comes with experience. Unfortunately, there is not any methodology in the literature that could guide a human to design the reward function or to allow a human to transfer the skills developed in designing reward functions to another human and in a systematic manner. In this paper, we use Systematic Instructional Design, an approach in human education, to engineer a machine education methodology to design reward functions for reinforcement learning. We demonstrate the methodology in designing a hierarchical genetic reinforcement learner that adopts a neural network representation to evolve a swarm controller for an agent shepherding a boids-based swarm. The results reveal that the methodology is able to guide the design of hierarchical reinforcement learners, with each model in the hierarchy learning incrementally through a multi-part reward function. The hierarchy acts as a decision fusion function that combines the individual behaviours and skills learnt by each instruction to create a smart shepherd to control the swarm.