Averaging Gone Wrong: Using Time-Aware Analyses to Better Understand Behavior

TL;DR

This paper emphasizes the importance of time-aware analyses in understanding online community behavior, revealing how user activity and engagement evolve over time and how ignoring temporal factors can lead to misinterpretations.

Contribution

It introduces a temporal analysis framework applied to Reddit data, demonstrating how time-aware methods uncover nuanced user behavior patterns and prevent misleading conclusions.

Findings

User behavior varies significantly across different time cohorts.

Ignoring time can lead to misinterpretation of trends, such as Simpson's Paradox.

Longer-lived users tend to be more active and make more, shorter comments.

Abstract

Online communities provide a fertile ground for analyzing people's behavior and improving our understanding of social processes. Because both people and communities change over time, we argue that analyses of these communities that take time into account will lead to deeper and more accurate results. Using Reddit as an example, we study the evolution of users based on comment and submission data from 2007 to 2014. Even using one of the simplest temporal differences between users---yearly cohorts---we find wide differences in people's behavior, including comment activity, effort, and survival. Further, not accounting for time can lead us to misinterpret important phenomena. For instance, we observe that average comment length decreases over any fixed period of time, but comment length in each cohort of users steadily increases during the same period after an abrupt initial drop, an example of Simpson's Paradox. Dividing cohorts into sub-cohorts based on the survival time in the community provides further insights; in particular, longer-lived users start at a higher activity level and make more and shorter comments than those who leave earlier. These findings both give more insight into user evolution in Reddit in particular, and raise a number of interesting questions around studying online behavior going forward.