Rally and WebScience: A Platform and Toolkit for Browser-Based Research on Technology and Society Problems
Anne Kohlbrenner, Ben Kaiser, Kartikeya Kandula, Rebecca Weiss,, Jonathan Mayer, Ted Han, Robert Helmer

TL;DR
This paper introduces Rally and sdkName, a platform and toolkit designed to facilitate browser-based research on technology and society issues, addressing methodological limitations of traditional approaches.
Contribution
It presents a new platform and toolkit that lower barriers for deploying browser-based studies, enabling more ecologically valid research in online environments.
Findings
Successful deployment of a pilot study with over 4 million webpage visits
Identification of implementation barriers in current browser-based research methods
Positive feedback from researchers using the new systems
Abstract
Empirical technology and society research is in a methodological crisis. Problems increasingly involve closed platforms, targeted content, and context-specific behavior. Prevailing research methods, such as surveys, tasks, and web crawls, pose design and ecological validity limitations. Deploying studies in participant browsers and devices is a promising direction. These vantage points can observe individualized experiences and implement UI interventions in real settings. We survey scholarship that uses these methods, annotating 284 sampled papers. Our analysis demonstrates their potential, but also recurring implementation barriers and shortcomings. We then present Rally and sdkName, a platform and toolkit for browser-based research. These systems lower implementation barriers and advance the science of measuring online behavior. Finally, we evaluate Rally and sdkName against…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsImpact of Technology on Adolescents · Social Media and Politics · Innovative Human-Technology Interaction
