Estimating abilities with an Elo-informed growth model

TL;DR

This paper introduces an Elo-informed growth model for estimating student abilities in intelligent tutoring systems, allowing dynamic ability tracking based on practice exercises rather than traditional static tests.

Contribution

It proposes a novel ability estimation method that accounts for changing abilities over time, improving accuracy over standard Elo algorithms in learning contexts.

Findings

Outperforms standard Elo in cases of rapid ability change.

Effectively estimates abilities without assuming a fixed growth curve.

Handles irregular time intervals between assessments.

Abstract

An intelligent tutoring system (ITS) aims to provide instructions and exercises tailored to the ability of a student. To do this, the ITS needs to estimate the ability based on student input. Rather than including frequent full-scale tests to update our ability estimate, we want to base estimates on the outcomes of practice exercises that are part of the learning process. A challenge with this approach is that the ability changes as the student learns, which makes traditional item response theory (IRT) models inappropriate. Most IRT models estimate an ability based on a test result, and assume that the ability is constant throughout a test. We review some existing methods for measuring abilities that change throughout the measurement period, and propose a new method which we call the Elo-informed growth model. This method assumes that the abilities for a group of respondents who are all in the same stage of the learning process follow a distribution that can be estimated. The method does not assume a particular shape of the growth curve. It performs better than the standard Elo algorithm when the measured outcomes are far apart in time, or when the ability change is rapid.