Who will stay? Using Deep Learning to predict engagement of citizen scientists

TL;DR

This paper develops deep learning models to predict citizen scientists' future engagement based on their activity patterns, aiding targeted interventions to sustain volunteer participation in environmental monitoring.

Contribution

It introduces the use of Deep Neural Networks with annotation sequences to forecast volunteer retention, a novel approach in citizen science engagement prediction.

Findings

Models can predict if volunteers will stay or leave.

Engagement metrics based on activity patterns are effective.

Potential for improved predictions with user profile data.

Abstract

Citizen science and machine learning should be considered for monitoring the coastal and ocean environment due to the scale of threats posed by climate change and the limited resources to fill knowledge gaps. Using data from the annotation activity of citizen scientists in a Swedish marine project, we constructed Deep Neural Network models to predict forthcoming engagement. We tested the models to identify patterns in annotation engagement. Based on the results, it is possible to predict whether an annotator will remain active in future sessions. Depending on the goals of individual citizen science projects, it may also be necessary to identify either those volunteers who will leave or those who will continue annotating. This can be predicted by varying the threshold for the prediction. The engagement metrics used to construct the models are based on time and activity and can be used to infer latent characteristics of volunteers and predict their task interest based on their activity patterns. They can estimate if volunteers can accomplish a given number of tasks in a certain amount of time, identify early on who is likely to become a top contributor or identify who is likely to quit and provide them with targeted interventions. The novelty of our predictive models lies in the use of Deep Neural Networks and the sequence of volunteer annotations. A limitation of our models is that they do not use embeddings constructed from user profiles as input data, as many recommender systems do. We expect that including user profiles would improve prediction performance.