A Framework for Learning Assessment through Multimodal Analysis of Reading Behaviour and Language Comprehension

TL;DR

This paper presents a multimodal framework for assessing reading comprehension and language skills in second language learners, analyzing eye-tracking, writing, and speaking data to evaluate the impact of repeated reading practices.

Contribution

It introduces a novel approach combining multimodal data and machine learning to automatically assess comprehension and the effects of reading frequency, which was not previously explored.

Findings

Multimodal skills can predict comprehension levels.

Repeated reading improves reading, writing, and speaking fluency.

Significant features identified for automatic assessment.

Abstract

Reading comprehension, which has been defined as gaining an understanding of written text through a process of translating grapheme into meaning, is an important academic skill. Other language learning skills - writing, speaking and listening, all are connected to reading comprehension. There have been several measures proposed by researchers to automate the assessment of comprehension skills for second language (L2) learners, especially English as Second Language (ESL) and English as Foreign Language (EFL) learners. However, current methods measure particular skills without analysing the impact of reading frequency on comprehension skills. In this dissertation, we show how different skills could be measured and scored automatically. We also demonstrate, using example experiments on multiple forms of learners' responses, how frequent reading practices could impact on the variables of multimodal skills (reading pattern, writing, and oral fluency). This thesis comprises of five studies. The first and second studies are based on eye-tracking data collected from EFL readers in repeated reading (RR) sessions. The third and fourth studies are to evaluate free-text summary written by EFL readers in repeated reading sessions. The fifth and last study, described in the sixth chapter of the thesis, is to evaluate recorded oral summaries recited by EFL readers in repeated reading sessions. In a nutshell, through this dissertation, we show that multimodal skills of learners could be assessed to measure their comprehension skills as well as to measure the effect of repeated readings on these skills in the course of time, by finding significant features and by applying machine learning techniques with a combination of statistical models such as LMER.