Daily Turking: Designing Longitudinal Daily-task Studies on Mechanical Turk
Henry Turner, Simon Eberz, Ivan Martinovic

TL;DR
This paper presents a system for conducting longitudinal daily-task studies on Amazon Mechanical Turk, focusing on touch dynamics, worker engagement, and study design challenges, demonstrating its viability as an alternative to lab studies.
Contribution
It introduces a novel system for long-term daily-task studies on Mechanical Turk and evaluates key factors affecting worker retention and data quality over time.
Findings
Longitudinal studies on Mechanical Turk are feasible and effective.
Payment structure and reminders influence worker engagement.
Informed consent can be balanced with quick task completion.
Abstract
In this paper, we present our system design for conducting longitudinal daily-task studies with the same workers throughout on Amazon Mechanical Turk. We implement this system to conduct a study into touch dynamics, and present our experiences, challenges and lessons learned from doing so. Study participants installed our application on their Apple iOS phones and completed two tasks daily for 31 days. Each task involves performing a series of scrolling or swiping gestures, from which behavioral information such as movement speed or pressure is extracted. The completion of the daily tasks did not require extra interaction with the Mechanical Turk platform, yet paid workers through it. This differs somewhat from the typical rapid completion of one-off tasks that workers are used to on Amazon Mechanical Turk. This atypical use of the platform prompted us to evaluate aspects related to…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsUser Authentication and Security Systems · Digital Mental Health Interventions · Mobile Crowdsensing and Crowdsourcing
