Talking by the numbers: Networks identify productive forum discussions

TL;DR

This study uses network analysis on forum logs from an introductory physics course to identify how collaboration structures relate to student success, revealing that network centrality correlates with final grades especially in dense discussion networks.

Contribution

It introduces a network-based approach to analyze forum discussions, demonstrating the link between network centrality and student performance, and highlights the importance of 'noise' in network signals.

Findings

Network density varies significantly between semesters.

Higher network centrality correlates with better course grades.

Backbone extraction removes meaningful signals, affecting correlation analysis.

Abstract

Discussion forums provide a channel for students to engage with peers and course material outside of class, accessible even to commuter and non-traditional populations. As such, forums can build classroom community as well as aid learning, but students do not always take up these tools. We use network analysis to compare three semesters of forum logs from an introductory calculus-based physics course. The networks show dense structures of collaboration that differ significantly between semesters, even though aggregate participation statistics remain steady. After characterizing network structure for each semester, we correlate centrality with final course grade. Finally, we use a backbone extraction procedure to clean up "noise" in the network and clarify centrality/grade correlations. We find that network centrality is positively linked with course success in the two semesters with denser forum networks, and is a more reliable indicator than non-network measures such as post count. Backbone extraction destroys these correlations, suggesting that the "noise" is in fact signal and further analysis of the discussion transcripts is required.