RelAItionship Building: Analyzing Recruitment Strategies for Participatory AI

TL;DR

This paper analyzes recruitment strategies in Participatory AI projects, highlighting challenges and providing recommendations to improve stakeholder engagement and ensure inclusive, equitable AI development.

Contribution

It offers a comprehensive analysis of current recruitment practices in Participatory AI through a corpus study and researcher interviews, proposing relationship-focused recruitment methods.

Findings

Recruitment outcomes are influenced by structural work conditions.

Researchers' goals and expectations shape recruitment success.

Effective recruitment fosters better collaboration and inclusivity.

Abstract

Participatory AI, in which impacted community members and other stakeholders are involved in the design and development of AI systems, holds promise as a way to ensure AI is developed to meet their needs and reflect their values. However, the process of identifying, reaching out, and engaging with all relevant stakeholder groups, which we refer to as recruitment methodology, is still a practical challenge in AI projects striving to adopt participatory practices. In this paper, we investigate the challenges that researchers face when designing and executing recruitment methodology for Participatory AI projects, and the implications of current recruitment practice for Participatory AI. First, we describe the recruitment methodologies used in AI projects using a corpus of 37 projects to capture the diversity of practices in the field and perform an initial analysis on the documentation of recruitment practices, as well as specific strategies that researchers use to meet goals of equity and empowerment. To complement this analysis, we interview five AI researchers to learn about the outcomes of recruitment methodologies. We find that these outcomes are shaped by structural conditions of their work, researchers' own goals and expectations, and the relationships built from the recruitment methodology and subsequent collaboration. Based on these analyses, we provide recommendations for designing and executing relationship-forward recruitment methods, as well as reflexive recruitment documentation practices for Participatory AI researchers.

Tables

Table 1. Table 1: Research Questions and Takeaways

Questions & Data	Takeaways
What are the recruitment strategies used in AI? Data: Corpus	• Decision users and data creators are often the ones recruited. • Researchers typically initiate recruitment. • Most recruitment is done via organizations and social media, websites, advertisements, and formal events. • Many details are not documented.
How do methods impact goals of Participatory AI? Data: Interviews	• Researchers’ capacities to engage in participatory projects are limited by structural conditions. • Researchers’ expectations and goals shape the methods they use and the relationships built with stakeholders. • Project outcomes depend on the relationships built by the researchers and participants.
How can we improve outcomes of recruitment methods? Data: Corpus, Interviews	• Relationship building is crucial when designing recruitment methodologies. • Building more processes and structural support, such as institutions that facilitate connections, can help improve recruitment methods. • Recruitment methodologies should be documented.

Table 2. Table 2: Non-exhaustive set of recruitment strategies in several fields

	Diverse and representative samples	Equitable relationships with communities
Computing	Oversample minoritized groups, work with communities, remove barriers to participation, build participant trust, and use inclusive demographic qualities (Roscoe 2021)	Work with community groups in developing recruitment strategies (University of Michigan STPP program and Detroit Disability Power and the Detroit Justice Center; 2024)
Civics	Email and postal mail (Sharp and Anderson 2010)	Reach out to grassroots organizations, community leaders and local media (Nabatchi et al. 2012)
Social Sciences	Participate in existing communities to gain access, snowball sample, consider participants’ perception of research (Hollingshead and Poole 2012)	Work with advocates and leaders of community groups (O’Brien et al. 2022)
Community Engagement	Develop culturally-specific engagement materials, work with community liaisons (Mancera et al. 2022)	Work with advocates and leaders of community groups, provide compensation (Beyond Chicago 2020)
Health and Medicine	Call and text using phone numbers from EHR (Langer et al. 2021)	Offer pay and accommodate meeting needs, and recruit through organizations, social/print media, clinics, snowball sampling (Leslie, Khayatzadeh-Mahani, and MacKean 2019)

Table 3. Table 3: Participants and descriptions (including Global South (GS) and/or Global North (GN) context focus)

ID	Role	Project experience discussed	Context	Academic Training
P1	Academia	AI for humanitarian purposes	GN	Computer Science
P2	Academia	AI for environmental and humanitarian purposes	GS & GN	Computer Science
P3	Academia	AI for humanitarian purposes	GS & GN	Computer Science
P4	nonprofit	AI and tech policy focused on marginalized groups’ concerns	GN	Information Sciences
P5	Academia	AI for humanitarian purposes	GN	Computer Science

Table 4. Table 4: What were the recruitment strategies?

Recruiting Strategy	Subcodes	Case studies
Organizational Recruitment (22)	Institutions (Universities); Online Communities; Crowdsourcing platforms; Organizations (community advocacy organizations); Community research groups; Monitoring Boards; Advisory Boards; Non governmental organizations; Social service providers and programs; Worker advocacy organizations; Employers and corporations	(Young, Magassa, and Friedman 2019; Suresh et al. 2022; Lee et al. 2019; Zhu et al. 2018; Newman et al. 2022; Cheng et al. 2021; Yu et al. 2020; Shi et al. 2020; Queerinai et al. 2023; Delgado, Barocas, and Levy 2022; Frauenberger et al. 2015; Imran et al. 2020; Eitzel et al. 2021; Sendak et al. 2020; UNHCR Innovation Service 2019; Wilder et al. 2021; Kuo et al. 2023; Flanigan et al. 2021; Calacci and Pentland 2022; Katell et al. 2020; Irani and Silberman 2013; Marian 2023)
Infrastructural Recruitment (17)	Social Media; Social media groups; Advertising; Radio; Websites; Talk Pages; Forums; Crowdsourcing platforms; News Articles; Publications; Loudspeaker announcements; Posters; Slack channels; Email listservs; Press Coverage; Bicycle shops; Bus stops	(Nekoto et al. 2020; Zhu et al. 2018; Freedman et al. 2020; Newman et al. 2022; Cheng et al. 2021; Yu et al. 2020; Puussaar et al. 2022; Shen et al. 2022; Halfaker and Geiger 2020; Li et al. 2023; Awad et al. 2018; Bakker et al. 2022; Deng et al. 2009; Swanson et al. 2015; Calacci and Pentland 2022; Kelling et al. 2012; Irani and Silberman 2013)
Personal Networks (10)	Individual Networks; Volunteering; House to House knocking; Snowball Sampling; Direct contacts; Word-of-mouth; Cold email; Attending summer schools and programs	(Nekoto et al. 2020; Puussaar et al. 2022; Shen et al. 2022; Halfaker and Geiger 2020; Eitzel et al. 2021; Kuo et al. 2023; Calacci and Pentland 2022; Kelling et al. 2012; Irani and Silberman 2013; Wilder et al. 2021)
Events (6)	Professional conferences; Civic meetups; Trainings; Village Meetings; Exploratory Workshops; Workshops; Networking Events; Presentations	(Nekoto et al. 2020; Puussaar et al. 2022; Queerinai et al. 2023; Delgado, Barocas, and Levy 2022; Sendak et al. 2020; Irani and Silberman 2013)
Not Documented for at least one participant group		(Finn et al. 2022; Liao and Muller 2019; Sinders 2020; Imran et al. 2020; Bondi et al. 2018)

Table 5. Table 5: Who initiated the recruitment?

Who initiated the recruitment strategy?	Case Studies
Researchers (24)	(Young, Magassa, and Friedman 2019; Lee et al. 2019; Zhu et al. 2018; Freedman et al. 2020; Newman et al. 2022; Cheng et al. 2021; Yu et al. 2020; Puussaar et al. 2022; Liao and Muller 2019; Shen et al. 2022; Frauenberger et al. 2015; Eitzel et al. 2021; UNHCR Innovation Service 2019; Wilder et al. 2021; Awad et al. 2018; Bakker et al. 2022; Deng et al. 2009; Swanson et al. 2015; Kuo et al. 2023; Katell et al. 2020; Marian 2023; Irani and Silberman 2013; Sinders 2020; Kelling et al. 2012)
Participants (5)	(Delgado, Barocas, and Levy 2022; Eitzel et al. 2021; UNHCR Innovation Service 2019; Calacci and Pentland 2022; Sendak et al. 2020)
No clear delineation between the two roles (4)	(Nekoto et al. 2020; Queerinai et al. 2023; Li et al. 2023; Halfaker and Geiger 2020)
Unclear for at least one participant group (6)	(Finn et al. 2022; Suresh et al. 2022; Shi et al. 2020; Imran et al. 2020; Flanigan et al. 2021; Calacci and Pentland 2022)

Table 6. Table 7: Are all the stakeholder recruitment strategy documented?

Value	Case Studies
All stakeholders have recruitment strategies documented	(Nekoto et al. 2020; Young, Magassa, and Friedman 2019; Lee et al. 2019; Zhu et al. 2018; Freedman et al. 2020; Cheng et al. 2021; Yu et al. 2020; Queerinai et al. 2023; Shen et al. 2022; Halfaker and Geiger 2020; Sendak et al. 2020; UNHCR Innovation Service 2019; Li et al. 2023; Awad et al. 2018; Bakker et al. 2022; Kuo et al. 2023; Irani and Silberman 2013)
Not all stakeholders have recruitment strategies documented	(Finn et al. 2022; Suresh et al. 2022; Newman et al. 2022; Puussaar et al. 2022; Shi et al. 2020; Liao and Muller 2019; Delgado, Barocas, and Levy 2022; Sinders 2020; Frauenberger et al. 2015; Imran et al. 2020; Eitzel et al. 2021; Wilder et al. 2021; Deng et al. 2009; Swanson et al. 2015; Flanigan et al. 2021; Calacci and Pentland 2022; Kelling et al. 2012; Katell et al. 2020; Bondi et al. 2018; Marian 2023)

Table 7. Table 8: Delgado’s Empowerment Code

Empowerment Code	Papers
Collaborate	(Katell et al. 2020; Wood et al. 2011; Calacci and Pentland 2022; Wilder et al. 2021; Halfaker and Geiger 2020; Delgado, Barocas, and Levy 2022; Shi et al. 2020; Queerinai et al. 2023; Finn et al. 2022; Lee et al. 2019; Suresh et al. 2022)
Consult	(Awad et al. 2018; Swanson et al. 2015; Bakker et al. 2022; Imran et al. 2020; Liao and Muller 2019; UNHCR Innovation Service 2019; Newman et al. 2022; Freedman et al. 2020; Cheng et al. 2021)
Include	(Marian 2023; Kuo et al. 2023; Bakker et al. 2022; Irani and Silberman 2013; Kelling et al. 2012; Calacci and Pentland 2022; Sendak et al. 2020; Sinders 2020; Imran et al. 2020; Shen et al. 2022; Frauenberger et al. 2015; UNHCR Innovation Service 2019; Yu et al. 2020; Queerinai et al. 2023; Zhu et al. 2018; Young, Magassa, and Friedman 2019)
Other (Not documented or Difficult to Apply the scale)	(Bondi et al. 2018; Marian 2023; Kuo et al. 2023; Swanson et al. 2015; Flanigan et al. 2021; Deng et al. 2009; Calacci and Pentland 2022; Wilder et al. 2021)
Own	(Kuo et al. 2023; Calacci and Pentland 2022; Eitzel et al. 2021; Sinders 2020; Wilder et al. 2021; Li et al. 2023; Delgado, Barocas, and Levy 2022; Puussaar et al. 2022; Suresh et al. 2022; Nekoto et al. 2020)

Table 8. Table 9: How specific are the institutions, platforms, and organizations?

Specificity	Papers
By Name	(Nekoto et al. 2020; Suresh et al. 2022; Lee et al. 2019; Freedman et al. 2020; Liao and Muller 2019; UNHCR Innovation Service 2019; Eitzel et al. 2021; Sendak et al. 2020; Imran et al. 2020; Awad et al. 2018; Deng et al. 2009; Swanson et al. 2015; Flanigan et al. 2021; Calacci and Pentland 2022; Kelling et al. 2012; Katell et al. 2020; Bondi et al. 2018; Marian 2023) (18)
By Kind	(Finn et al. 2022; Young, Magassa, and Friedman 2019; Newman et al. 2022; Cheng et al. 2021; Sinders 2020; Frauenberger et al. 2015; Wilder et al. 2021; Bakker et al. 2022; Kuo et al. 2023) (9)
By Kind and Name	(Zhu et al. 2018; Yu et al. 2020; Puussaar et al. 2022; Shi et al. 2020; Queerinai et al. 2023; Delgado, Barocas, and Levy 2022; Shen et al. 2022; Halfaker and Geiger 2020; Li et al. 2023; Irani and Silberman 2013) (10)

Table 9. Table 10: Table of Numbers by Country

Country	Number
Botswana	1
Ethiopia	1
Global	10
Netherlands	1
New Zealand	1
South Africa	1
Tanzania	1
Uganda	1
United Kingdom	6
United States	17
Uruguay	1
Zimbabwe	2

Table 10. Table 11: Domains

Category	Count
Low resourced languages	2
Tech Policy	2
Social Justice	1
Humanitarian	4
Wikipedia	3
Health	3
Agriculture	2
Public Services	2
Civics	2
LGBTQIA+	2
AI Governance	3
Law	1
Education	2
Social Media	1
Computer Vision	1
Conservation	3
Housing	1
Labor	2

Table 11. Table 12: Participation Entry Point

Category	Count
All points	11
Dataset Development	10
Deployment and Monitoring	11
Problem Formulation	8
Model Design and Training	5
N/A	1

Table 12. Table 13: Citations of Corpus and Corresponding Links

Citation	Link
Awad et al. (2018)	https://www.nature.com/articles/s41586-018-0637-6
Bakker et al. (2022)	https://proceedings.neurips.cc/paper˙files/paper/2022/file/f978c8f3b5f399cae464e85f72e28503-Paper-Conference.pdf
Bondi et al. (2018)	https://www.cais.usc.edu/wp-content/uploads/2017/11/spot-camera-ready.pdf
Calacci and Pentland (2022)	https://dl.acm.org/doi/10.1145/3570601
Cheng et al. (2021)	https://dl.acm.org/doi/fullHtml/10.1145/3411764.3445308
Delgado, Barocas, and Levy (2022)	https://dl.acm.org/doi/10.1145/3512898
Deng et al. (2009)	https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5206848
Eitzel et al. (2021)	https://theoryandpractice.citizenscienceassociation.org/articles/10.5334/cstp.339
Finn et al. (2022)	https://aclanthology.org/2022.computel-1.12.pdf
Flanigan et al. (2021)	https://www.nature.com/articles/s41586-021-03788-6
Frauenberger et al. (2015)	https://www.sciencedirect.com/science/article/pii/S1071581914001232
Freedman et al. (2020)	https://arxiv.org/abs/2005.09755
Halfaker and Geiger (2020)	https://arxiv.org/pdf/1909.05189
Imran et al. (2020)	https://arxiv.org/abs/2004.06675
Irani and Silberman (2013)	https://escholarship.org/content/qt10c125z3/qt10c125z3˙noSplash˙17ace960c310470e904a7512eccf6ee2.pdf
Katell et al. (2020)	https://dl.acm.org/doi/10.1145/3351095.3372874
Kelling et al. (2012) 333Same case study as (Wood et al. 2011)	https://ojs.aaai.org/index.php/AAAI/article/view/18963
Kuo et al. (2023)	https://dl.acm.org/doi/10.1145/3544548.3580882
Lee et al. (2019)	https://dl.acm.org/doi/10.1145/3359283
Li et al. (2023)	https://dl.acm.org/doi/10.1145/3544548.3582074
Liao and Muller (2019)	https://arxiv.org/abs/1912.07381
Marian (2023)	https://dl.acm.org/doi/pdf/10.1145/3593013.3594009
Nekoto et al. (2020)	https://arxiv.org/abs/2010.02353
Newman et al. (2022)	https://ai4sibook.org/kudu/
Puussaar et al. (2022)	https://dl.acm.org/doi/abs/10.1145/3555215
Queerinai et al. (2023)	https://dspace.mit.edu/handle/1721.1/151097
Sendak et al. (2020)	https://dl.acm.org/doi/10.1145/3351095.3372827
UNHCR Innovation Service (2019)	https://medium.com/unhcr-innovation-service/a-goat-story-3ed6bdd2b237
Shen et al. (2022)	https://dl.acm.org/doi/pdf/10.1145/3531146.3533110
Shi et al. (2020)	https://ai4sibook.org/wp-content/uploads/2022/08/Food-Rescue.pdf
Sinders (2020)	https://carolinesinders.com/wp-content/uploads/2020/05/Feminist-Data-Set-Final-Draft-2020-0526.pdf
Suresh et al. (2022)	https://dl.acm.org/doi/pdf/10.1145/3531146.3533132
Swanson et al. (2015)	https://www.nature.com/articles/sdata201526
Wilder et al. (2021)	https://ai4sibook.org/hiv/
Wood et al. (2011) 444Same case study as (Kelling et al. 2012)	https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1001220
Young, Magassa, and Friedman (2019)	https://link.springer.com/article/10.1007/s10676-019-09497-z
Yu et al. (2020)	https://dl.acm.org/doi/10.1145/3357236.3395528
Zhu et al. (2018)	https://dl.acm.org/doi/10.1145/3274463

Full text

1 Introduction

Artificial intelligence for social impact (AISI) is being developed and deployed in many domains with promise to address pressing societal challenges, including healthcare (Kamineni et al. 2023) and conservation (Tuia et al. 2022). However, without involving stakeholders that are potentially affected by AI, there are risks to deploying AI broadly, such as encoding racism into AI systems (Benjamin 2019; Obermeyer et al. 2019) or supporting the centralization of power (Birhane et al. 2022b). Using participatory methods to carefully engage with stakeholders and impacted community members while developing AI systems realizes values of collective governance and socially beneficial AI (Bondi et al. 2021; Birhane et al. 2022a; Costanza-Chock 2020a). A body of work has been referring to the incorporation of participatory methods, drawn from theories of participatory design in computing and beyond, into AI development as “Participatory AI” (Kulynych et al. 2020).

This ethos of participation, while shared by many who study or aim to implement AISI and Participatory AI, can be difficult to achieve in practice (Lin et al. 2024). Barriers to implementing Participatory AI include a lack of clear standards and best practices (Delgado et al. 2023). Also, while Participatory AI idealizes the inclusion of marginalized and impacted communities, AI practitioners report difficulties in recruiting and engaging with certain impacted communities (Cooper and Zafiroglu 2024; Kallina and Singh 2024). Despite the crucial impact of recruitment on who participates (Vines et al. 2013), there is no systematic practical guidance on recruitment methods that AI researchers can use, especially for those who are new to participatory methods.

We aim to develop practical guidance on recruitment strategies by studying the current practice of AI researchers. Our main research questions in this work are: 1) What are the recruitment strategies used by AI researchers? 2) How do methods impact goals of Participatory AI, and 3) How can we improve outcomes of recruitment methods? First, we review a corpus of 37 AI projects to characterize and describe the current recruitment strategies used in AI, and note that recruitment methods are often not documented sufficiently for purposes of reproducibility and transparency. Then, we interview five AI researchers and practitioners to learn what methods tend to be more successful and why. By analyzing both our corpus data and the interviews, we learn that structural conditions, researchers’ own goals and expectations for the project, and relationships between researchers and community groups are key factors that influence the outcomes and the kinds of impact accomplished by the AI system (Table 1). We then introduce recommendations to improve the outcomes of recruitment methodology towards achieving Participatory AI.

In the rest of the paper, we will provide an overview of related work in human-centered design and Participatory AI, discuss the methods we used for our review of existing practices of participation and our semi-structured interviews, the results of our analyses, and then introduce our recommendations for the design of recruitment strategies.

2 Related Work

2.1 Participatory Methods in Computing

Participation as a research methodology in computing draws from the fields of participatory design (Spinuzzi 2005; Asaro 2000) and participatory action research (Costanza-Chock 2020a). Participatory approaches’ focus on participant knowledge and deconstructing knowledge hierarchies (Frauenberger et al. 2015) lends itself well to design methodologies that aim to achieve socially just outcomes by elevating marginalized voices. Because of this, participatory approaches are key components to design methodologies in computing oriented towards social justice (Abebe et al. 2022; Irani et al. 2010; Bardzell and Bardzell 2011), and calls to apply principles from participatory research traditions to AI are increasing (Klein and D’Ignazio 2024; Costanza-Chock 2020b; Mohamed, Png, and Isaac 2020).

At the same time, participatory methods in AI development have been critiqued for “participation washing,” in which participatory methods benefit the designers at the expense of the participants and manufacture consent for decisions made undemocratically (Sloane et al. 2022). In response to these critiques, scholars have worked to empirically characterize current Participatory AI practice (Cooper and Zafiroglu 2024; Groves et al. 2023; Kallina, Bohné, and Singh 2025), develop conceptual frameworks to evaluate existing participatory projects in AI according to the goals of participant empowerment and democratizing AI governance (Birhane et al. 2022a; Sloane et al. 2022; Berditchevskaia, Peach, and Malliaraki 2021; Feffer et al. 2023; Bondi et al. 2021; Corbett, Denton, and Erete 2023; Kallina, Bohné, and Singh 2025), and provide high level guidance and values to design participatory methods to achieve these goals (Berditchevskaia, Peach, and Malliaraki 2021; Feffer et al. 2023; Delgado et al. 2023; Gilman 2023; Bondi et al. 2021). The critical questions raised by previous work indicate that both practice and the social context of participatory methods are important in determining whether Participatory AI results in equitable, democratic AI development and systems.

2.2 Recruitment in Participatory AI

A key concern in Participatory AI is studying how technology designers shape the participatory process. Cooper and Zafiroglu (2024) examine the practices that designers, dubbed “participation brokers,” use to shape who participates and how they participate in AI design. They note the degree of power that AI researchers have in determining these practices, and observe that AI researchers frequently form partnerships with community groups (Cooper and Zafiroglu 2024). In another vein, Lin et al. (2024) studies how AISI partnerships often fail to meet the needs of community partners due to issues such as funding constraints and mismatched expectations of AI’s capabilities. They suggest that AI designers follow the lead of community members early in the project for AISI projects (Lin et al. 2024). Power and values-based recommendations for designing participatory methods are other key points (Costanza-Chock 2020a; Harrington, Erete, and Piper 2019).

Yet, there is still a need to provide practical guidance for participatory methods in AI. Kallina and Singh (2024) interview AI practitioners in corporate and organizational contexts, finding that 70% of respondents reported a lack of inclusivity in their samples (Kallina and Singh 2024). Respondents also note that certain stakeholder groups are difficult to identify and contact, in addition to having low incentives to participate. Other AI practitioners in corporate contexts similarly report needing clearer guidance as well as incentive structures to put participatory methods into practice (Delgado et al. 2023; Groves et al. 2023; Kallina, Bohné, and Singh 2025). As these works note, recruitment is key for achieving goals of Participatory AI; it configures who participates in the first place, and can impact the goals of sharing design control with impacted communities (Vines et al. 2013) and the values embedded in and performance of machine learning models (Diaz and Smith 2024). We aim to contribute to this body of work by clarifying the practice of participatory methods, particularly for recruitment.

2.3 Recruitment Methods

Existing recruitment methodologies seek to accomplish primary research goals, such as conducting surveys and interviews (Heerman et al. 2017), participant observation and other ethnographic methods (Negrin et al. 2022), co-design practices, and more. Other methodologies, drawn from research traditions such as participatory action research, also ensure diverse and representative sampling, encouraging equitable relationships with communities, and building organizational capacity (University of Michigan STPP program and Detroit Disability Power and the Detroit Justice Center; 2024; O’Brien et al. 2022; Beyond Chicago 2020). We refer to Table 2 for a brief overview of existing recruitment methods for a variety of purposes. Social scientists interviewing online communities, for example, highlight strategies such as participating in existing communities, using snowball sampling, and considering whether participants want to participate in the research projects (Hollingshead and Poole 2012). This suggests approaches to recruitment that are relational and dependent on the researcher’s positionality.

Reflection on recruitment in the context of researchers’ specific positionalities and power imbalances is also prevalent. For example, relationships and institutional context impact recruitment, the research questions that are pursued, and participant trust in researchers (Le Dantec and Fox 2015; Small and Calarco 2022; Gibson and Abrams 2003). In particular, Sum et al. (2025) examine how exploitative relationships between the university and local communities influence researchers’ partnerships with community-based organizations. They ask researchers to build sustained relationships with community members, adopt individual tactics to support communities, and encourage institutions to revest in communities (Sum et al. 2025). Similarly, recruitment methods for non-“WEIRD” populations must also be contextual, flexible, and focus on needs and existing technological practices to be successful (De, Kanthawala, and Maddox 2025).

In short, much recruitment literature exists that AI researchers can draw on. Yet, little work has studied recruitment methods in AI or how those methods impact Participatory AI goals. We aim to help fill these gaps, towards improving recruitment methodology for Participatory AI.

3 Methods

We investigated recruitment practices in projects that use participatory methods during the design and development of AI by: 1) creating and analyzing a corpus of 37 projects in this space; 2) conducting institutional review board-approved semi-structured interviews with 5 AI researchers.

3.1 Corpus Creation and Analysis

Assembling the Corpus

We conducted a literature review in order to build a corpus of AI projects that represented a diversity of modes of participation, as defined by Sloane et al. (2022). We drew case studies from existing reviews of Participatory AI, particularly (Birhane et al. 2022a; Delgado et al. 2023; Feffer et al. 2023; Berditchevskaia, Peach, and Malliaraki 2021; Kulynych et al. 2020), case studies of AI for Social Impact (Tambe, Fang, and Wilder 2022), recent publications at venues like FAccT, EAAMO, CHI, and AIES, and searches on Google Scholar for related terms like “Participatory AI.” We adopted this approach in an effort to include projects using a wide variety of participatory methods not explicitly labeled as such, including projects where participation served a variety of goals (e.g. participation as labor vs.  consultation (Sloane et al. 2022)) and impacts (e.g.  participant empowerment (Delgado et al. 2023)). Appendix111https://realize-lab.github.io/participaite Tables 10, 11, and 12 provide an overview of the countries represented by our sampled projects, the domains they were in, and the participation entry point (as defined by (Corbett, Denton, and Erete 2023)), respectively. To summarize, our corpus includes a wide variety of domains and both Global South and Global North contexts, with a majority of projects coming from Global North contexts. In addition, the majority of participation took place during “dataset development” and “deployment and monitoring” entry points. We determined saturation was reached and concluded the literature review when novel recruitment categories in AI projects were no longer identified.

Analyzing the corpus

After compiling the corpus, we used inductive coding (Braun and Clarke 2006) with a goal to explore high-level themes and categories to characterize the recruitment methodologies used by AI researchers and practitioners. We found broad categories of recruitment methods, stakeholder types, and the actors who initiated the recruitment. We also coded each project by the level of empowerment afforded to the participants in the project (Delgado et al. 2023), to help us explore the question of how recruitment methods impact goals of Participatory AI. Two authors (EK, BK) coded the papers. Due to the small number of coders, we focused on building coding consensus via weekly meetings. Results are in Section 4.

3.2 ParticipAIte

We included the high-level categorizations from our qualitative coding into a preliminary database, which we call ParticipAIte222https://realize-lab.github.io/participaite, and populated it with papers from our corpus. It includes information on recruitment methodology, including: a list of involved stakeholders, the recruitment strategy, who initiated recruitment, the specific call for participation, and lessons learned (see Appendix B).

3.3 Semi-Structured Interviews

For the semi-structured interviews, we aimed to gather more detail about the recruitment methodologies surfaced in our corpus, particularly from AI researchers with prior experience in recruiting participants for AI projects. Follow-up studies should also incorporate community group perspectives, which are essential for a more complete analysis of the empowerment of community groups.

We contacted ten researchers that were known to us to be researchers with Participatory AI experience via emails and direct messages, and interviewed five of them. Our selection was prioritized by our goal to converse with researchers who may have varied experiences in recruitment methodology, including the domain, format of participation, and goal of the participatory process. We acknowledge that the selection and response of participants were biased by our existing personal networks, interests, and reputations within certain research communities. See Table 3 for an overview of participant backgrounds.

Interviews lasted an hour, and were conducted via the video conferencing program Zoom. All interviews were recorded and transcribed with interviewee permission. We provided $50 gift cards as compensation. The study was determined by the University of Michigan Institutional Review Board to be exempt from ongoing review (HUM00266324).

Interviews had two main components. For both components, participants had access to the ParticipAIte database, which contained an entry of one of their existing projects, as well as a project in our corpus. First, we asked about interviewees’ previous and current experience with recruitment methodology, using database entries describing their recruitment methodology that we assembled from existing work about the projects. We aimed to elicit detailed narratives behind their experiences of developing recruitment methodology. The second main component was to discuss another recruitment case study in order to discuss documentation practices that would help with reproducibility. Finally, we asked for open ended, unstructured feedback on documentation about recruitment. See Appendix A for full interview questions.

As with the corpus, we performed an inductive coding on the interviews. Following thematic analysis (Braun and Clarke 2006), we developed higher level themes by analyzing data in context with the entire dataset, as well as descriptive themes. The results are in Sections 5 and 6.

4 Recruitment Methodologies

Our corpus analysis was based on descriptive categories of recruitment methodologies, including: Who is recruited, who recruits, recruitment strategy, and recruitment impacts on participant empowerment. We also noted patterns in documentation. The Appendix lists the full corpus and analysis.

Who is recruited?

In general, projects in our corpus tended to recruit either end users of an AI system (“Decision users”) or data labellers and workers (“Data creators”) as participants. Results are presented in Fig.1 and Table 6 in the Appendix.

Works in the corpus also sometimes detailed the relevant stakeholder groups that they did not aim to recruit from (Lee et al. 2019; Yu et al. 2020; Newman et al. 2022; Cheng et al. 2021; Sendak et al. 2020), acknowledging how recruitment strategies determine who can or cannot participate. Some projects excluded stakeholders due to the challenges of engaging said stakeholders and their more indirect relationships to the system being developed, such as patients in a tool designed for healthcare workers (Sendak et al. 2020).

In our corpus, authors also noted that recruitment can shape barriers to participation. For example, enabling online participation and flexible scheduling were some strategies to ensure equitable participation and broader recruitment (Queerinai et al. 2023; Kuo et al. 2023; Lee et al. 2019). A lack of inclusion due to barriers could in turn lead to bias and generalization challenges (Awad et al. 2018; Calacci and Pentland 2022; Marian 2023; Bakker et al. 2022).

Who recruits?

In addition to differences in who was recruited, there were also different stakeholders doing the recruitment (see Appendix Table 6 and Fig. 2). AI researchers or practitioners most frequently conducted recruitment in our corpus. Furthermore, some of our corpus provided descriptions of how the researchers or practitioners’ previous experiences with the communities of their participants allowed access to research sites (Lee et al. 2019; Wilder et al. 2021; Eitzel et al. 2021). Our corpus also had a few rich descriptions of collaboration building between researchers and community partners (Delgado, Barocas, and Levy 2022; UNHCR Innovation Service 2019).

There were a few instances where the line between researcher and participant was blurred, as both parties engaged in the process of research (Queerinai et al. 2023; Nekoto et al. 2020). In our corpus, we also found several examples of participant-led research, where researchers were solicited by communities or developed the research problem in collaboration with communities (see Appendix for a list).

Recruitment Strategy

Recruitment strategies were also quite varied, as shown in Fig. 3 and Table 4 in the Appendix. Recruitment strategies included working with organizations (organizational), drawing on personal networks, using social media or other mass media (infrastructural), and attending events like conferences (Table 4 details specific strategies in these categories). We found that most recruitment in our corpus was based on connecting to organizations, for example, by reaching out to community advocacy organizations or nonprofit organizations.

Empowerment

Our corpus showcased several ways for recruitment methods to meet goals of equity and empowerment. For example, researchers ensured that their methods followed community norms via conversations with the community (Zhu et al. 2018), relevant domain experts (Kuo et al. 2023), and local institutions (Newman et al. 2022). Works in our corpus also aimed to build trust with marginalized communities by recruiting institutional actors that worked with these communities (Wilder et al. 2021; Kuo et al. 2023; Young, Magassa, and Friedman 2019; Frauenberger et al. 2015) and key organizers of communities (Calacci and Pentland 2022; Marian 2023).

While crowdsourcing has been noted as a form of participation that is considered labor, is more at the level of consulting or including (using Delgado et al. (2023)’s scale of participant empowerment), or offers a low amount of agency to participants (Nekoto et al. 2020), careful crowdsourcing can also empower community groups. A prominent example is Digital Workerism (Calacci 2022), where crowdsourcing was used to provide data for a worker-led campaign to investigate the impact of algorithmically determined wages on working conditions. In our corpus, researchers and practitioners used intentional strategies to reach marginalized communities during crowdsourcing, including decentralized and internet-based recruitment strategies to reach community members over large geographical divides (Queerinai et al. 2023; Nekoto et al. 2020), requiring few resources to recruit and participate (Li et al. 2023; Queerinai et al. 2023), and reaching out to strategic allies during the recruitment process (Nekoto et al. 2020; Irani and Silberman 2013). Offering some direct benefits to participants, such as easing existing workflow challenges (Kelling et al. 2012), or ensuring that the resulting data and knowledge production were of immediate benefit to participants (Li et al. 2023), were also important practices.

4.1 Documentation of Recruitment Strategies

So far, we have focused on those parts of recruitment methodologies that are documented in our corpus. However, projects varied in level of documentation. For example, Figs. 2 - 3 showed multiple projects lacking enough documentation for us to determine a categorization. Some projects described the specific actors and institutions they work with, while others only referred to the kind of institution. While omitting specific names to protect privacy is certainly an important consideration, we believe documentation of other aspects of recruitment are important for purposes like reproducibility and transparency. For instance, we noted several helpful examples where authors detailed reasons for excluding or including certain stakeholders, how researchers leveraged existing personal relationships during recruitment, or specific motivations for selecting certain recruitment strategies. These were not documented across the entire corpus. As such, we were motivated to interview AI researchers and learn about their experiences with recruitment methodology.

5 Results from Interviews

Our interviews focused on interviewees’ previous experience with recruitment. To briefly summarize, each of the interviewees identified as researchers engaging in recruitment. They were primarily concerned with recruiting collaborators, mostly community organizations, other grassroots advocacy and activist groups, and members of specific communities, for co-design projects. To meet these recruitment goals, interviewees largely began by identifying partners with experience and existing relationships with communities the interviewee wished to engage with as a starting point, and reaching out most frequently through personal networks and cold calling. Once they established a relationship, e.g., with a nonprofit or government agency, they then often partnered with organizations for many core tasks, including: surveying communities, recommending policy, and introducing interviewees to other stakeholders. This aligns with our findings from the corpus, which showed the prevalence of organizational and personal network recruitment (Fig. 3), and slightly differs from formal social science recruitment methods, such as surveying or random sampling.

We now analyze and present our detailed findings in three connected themes. First, we explore how researchers’ capacities to begin and engage in participatory projects are inherently limited by structural conditions. Second, we explore how researchers’ expectations and goals shape the methods they use, and how methods can impact the relationships built with stakeholders in the domain. Finally, we argue that outcomes of participatory projects are fundamentally based on the relationships built by the researchers and participants.

5.1 Structural Challenges to Recruitment

Before we begin to address the recruitment methodologies that are used by researchers, as well as their outcomes, we need to discuss what constraints and limitations researchers face.

Funding

A first barrier is funding. Researchers noted: “to support this kind of project is not as easy compared to say, defense and security,” pointing to a clear lack of funding support for projects supporting community-based organizations (P2). Not only do the researchers need to get funding for themselves, they also need to provide funding for their collaborators (P2). Funding for the participating community-based organization is particularly important, as participants, staff, and the organization need to be paid for their time and any additional coordinating labor (P4). One interviewee also addressed the fact that continued funding is a significant problem to achieving participatory projects with a long-term goal and focus: “there’s not a ton of funding available for maintenance of systems” (P3).

Time and effort

A second barrier is the time commitment added on from deciding to pursue a participatory project. For example, researchers discussed how participation from organizations and communities required senior researchers to “identify a problem at the right scope” (P2) for a student to work on, as well as prototyping and creating pilot experiments to determine operational feasibility (P5). Due to the considerable time investments required from both the AI practitioners and the involved stakeholders, both parties would need to commit seriously to the length of the project (P2).

Logistics and coordination

These challenges are compounded by logistical and legal concerns. P5, for example, was hampered by such issues when attempting to include a certain stakeholder group. Coordinating with communities and stakeholder groups can require considerable investment and project management, including overhead not necessary when working alone, such as additional meetings (P5), testing prototypes in the field (P5), and engaging in the domain (P2).

Ethical engagement and building trust

The entrenched power dynamics between researchers and community groups is another barrier. One researcher discussed the difficulty in gaining the trust of certain groups: “you’re always working in the wake of whatever the university that you work at has done to that community” (P4). Cognizance of structural harms and of ways that marginalized groups are often further marginalized by exploitative projects means that researchers need to evaluate whether the community groups and organizations that they are working with even have the capacity to participate in these projects, and if participation is putting an undue burden on the community groups (P4).

Lack of (relevant) expertise

Some researchers are not trained in engaging with and recruiting community members or groups. One researcher stated: “we just do not have any of the expertise needed to kind of navigate that world and do anything sensible” (P1) when it comes to directly outreaching to community members. In fact, some interviewees were not involved with community engagement or recruitment despite being involved in Participatory AI projects, as teams of researchers had split into community-engagement and technical intervention subteams, with the technical researchers relatively separated from decisions about engaging with the community (P1, P5).

Furthermore, technical expertise can also serve as a structural challenge in recruitment. Several interviewees had explicit expectations to develop specific technical contributions as a result of a collaboration due to career incentives. This presents a very serious trade-off for interviewees: “how much effort and time do you really want to spend on these AI for social good projects that may not fully leverage your technical expertise to work on?” They suggested that, for researchers interested in creating social impact without engaging in the additional labor required to actively involve community organizations and groups, foregoing recruitment and working on open-source projects may be more efficient for achieving social impact (P2).

5.2 Impacts of Challenges and Goals on Recruitment

These challenges, coupled with researcher goals, ultimately incentivized researchers to find partners as a means of finding participants. This, in turn, shaped recruitment methods.

Personal connections and partners

Working with organizations proved to be an effective strategy for researchers to engage in participatory projects, particularly with organizations that had both the capacity to work with researchers and were already embedded in the communities of interest. This helped address challenges with logistics, engagement, expertise, and trust, to name a few.

For example, P4 noted that organizations with a grassroots orientation enable better connection to membership and an ability to directly bring community members into the process of Participatory AI, while P1, P2, and P5 stressed the importance of having a community partner that already had connections to communities and the relevant expertise to work in these domains. P4 also found that going to the director of advocacy or programming at nonprofits was promising due to an indication that the organization had an increased capacity for collaboration. These existing connections to partners with capacity were significant in starting a collaboration.

At the same time, the structural barriers that made partnering with community groups more effective also led to challenges. P2 summarized the challenge as “I think in general, it’s not easy, because we have limited capacity, and these nonprofits also have their own agenda and priorities” (P2). Furthermore, one interviewee reflected how success is often determined by an intuition surrounding the quality of the relationship: “I do feel like a lot of recruitment happens in sort of…a ground-up way…a lot of it is vibes” (P3). Ultimately, “finding [any] collaborator is easier than finding a good collaborator” (P2).

All of our interviewees reported heavily relying on personal networks in their recruitment processes; including academic and personal networks. In fact, our interviewees reported that relying on personal networks led to successful academic outcomes (P1-P5).

Mixed success from cold calls

To find partners, the alternative to leaning on connections was cold calling potential partners. Cold calls, whether through email, phone, or direct messages, were reported to have mixed success rates - failures of collaborations starting from cold calls included no responses (P2, P4), or collaborations that were short-term or underwhelming (P2). P2 shared an anecdote where a close colleague emailed dozens of nonprofits in their local area, but were unable to successfully create a collaboration. While researchers admitted that they could not always know why a cold-call attempt failed, given that people are very busy and have other priorities, refusal to participate could indicate a deeper mistrust of researchers and the institutions they represent (P4), adding yet another difficulty to the process.

Establishing a relationship from cold calls is often easier when the researcher can lean on their existing reputation in the community (P2, P3). P4 said that “If someone’s not familiar with you, I think that some of this work comes across like weird marketing research … like getting a cold call from an organization that wants to pay you…” (P4). To address this challenge and to successfully get a response, interviewees suggested showing examples of previous existing projects (P2), emphasizing compensation (P4), and making a case that the community group has novel, important contributions to make (P4).

Challenges to achieve representativeness

All of our interviewees engaged with goals of representativeness when they decided how and who to recruit. For instance, researchers were motivated to use cold calls to bring in partners they otherwise would not. P4 reflected: “It would be a lot easier if we did only rely on our personal networks, because the [organizations] would be much more likely to participate”, as they may already see the value in the project, but “if you stay too close to home, then you’re just talking to the same people with the same interests. So often, it’s cold calls” (P4).

One key challenge is that capacity limits interviewees’ ability to achieve ideals of representativeness. Even though our interviewees aspired to include all relevant groups, this was not practically possible. One interviewee regretted not including one group of stakeholders, mentioning that even a small, biased sample would be better than having none (P5). The reasons cited by researchers for not including these specific stakeholder groups were a lack of capacity to navigate the logistical (e.g., institutional review board) obstacles to include these groups, noting that doing so would have added potentially years to the research study (P5).

Researchers also emphasized that representativeness is a priority throughout project design and development; questions of how to make participation “successful and sustainable” are not “specific to recruitment” (P3). The way people participate and to what degree they can influence the system, for instance, are similarly important. Recruitment is key for achieving this goal of representativeness throughout a project.

5.3 Relationship Impacts on Outcomes

The challenges in finding participants underscore significant trade-offs between pursuing and building relationships with community groups, and ensuring participatory projects can meet researchers’ career needs, such as milestones and publishing requirements. Given that these kinds of tensions are largely structural, we found that the interviewees responded by trying to build longer-term relationships that were mutually beneficial to community partners and researchers.

Long-term partnerships

Long-term partnerships were noted as having many benefits for interviewees and the partners they were working with. Ideally, when recruiting partners, researchers were looking for “long-term collaborations” (P2). Long-term relationships were noted as valued by community groups, who have made it clear to researchers that parachuting into community to produce research and leaving is not acceptable (P2). Other benefits included producing several research papers over a stand-alone application (P2) and establishing new governance relations between designers, governors, and community organizations (P4). Echoing concerns and critiques of exploitative projects, P4 described the frequently short-term nature of Participatory AI as something “arguably broken about Participatory AI.”

Interviewees in our study discussed several more successful strategies for building longer-lasting, durable relationships between researchers and their collaborators (P1-P3). For example, P1 talked about a project in which the nonprofit center benefited, as the conducted research provided important administrative tasks and helped provide benefits to the community that the nonprofit served, despite the center choosing not to use the algorithm the researcher created in day-to-day usage. This relationship was mediated by a senior researcher’s long-standing relationship with a collaborator in the domain who had years of experience collaborating with relevant nonprofits. In addition, P2 spoke to a model of relationship-building that was more scalable and accessible to junior researchers with fewer resources: senior researchers maintaining long-standing relationships with many partners and nonprofits to help scope projects for more junior researchers, and relying on existing long-term relationships between senior researchers and community partners. Thus, long-term relationships can be the basis of many separate projects, and provide continued impact for community partners.

Relationship challenges and deployment

Yet, long-term partnerships are difficult to achieve. One interviewee recalled a collaboration that broke down right before the deployment of an algorithm, despite the CEO of the nonprofit being very enthusiastic about the project in the initial stages. When reflecting on what could have been improved, the interviewee suggested that the researchers and the nonprofit needed an “explicit collaboration” from the beginning to avoid failure: “Maybe just having them attend way more meetings, even on the [technical] side, would help them feel even more like part of this.” Unfortunately, in this case, the algorithm did not get to be deployed. Three of our interviewees (P1, P2, P5) reported that the algorithmic interventions created for nonprofit partners were not still currently deployed after the researchers “handed off” the project to the organizations. This happened when the project would end, and the researchers would no longer be involved in the project, often moving to other universities (P1 P2, P5), or when the nonprofit would fundamentally alter the way they collect data, rendering the algorithm obsolete (P1, P2). The challenges maintaining relationships between researchers and community-based organizations during the development of the algorithm may have contributed to limited algorithmic impacts.

Outcomes beyond deployment

Sometimes, benefits to nonprofits or community partners existed beyond algorithmic impacts. Other benefits mentioned in our interviews included receiving funding (P2, P4), gaining legitimacy in the eyes of governments and other decision-makers (P3), establishing data collection practices initiated by researchers (P1), or even developing AI literacy (P2). This indicates, then, that truly impactful participation might not be measured in terms of algorithm deployment alone, but also holistically with respect to other benefits.

6 Discussion

Given the challenges we found to successfully recruiting for Participatory AI, there is a clear need for improved recruitment methodology. Design recommendations for Participatory AI from previous work can and should apply to recruitment methods, such as recommendations to examine and challenge power (D’ignazio and Klein 2023), to prioritize marginalized communities (Costanza-Chock 2020a), and to ensure research meaningfully benefits the involved communities (Erete, Rankin, and Thomas 2021). The interviewees (see Section 5) wanted to achieve these kinds of goals, but the challenges they faced were significant. We now discuss possible approaches to improving recruitment strategies.

6.1 Building Community

Our findings highlighted the highly relational aspect of recruitment. From researchers’ reputations to building trust to achieving representativeness, recruitment methodologies often succeeded or failed based on the relationships between researchers and communities.

We encourage researchers to expand conceptions of what is a legible recruitment strategy and who is able to use these strategies. For instance, in addition to finding partners, researchers can establish themselves within the networks of communities that they aim to work with by helping organize or attending workshops that focus on prominent issues in local communities, or regularly volunteering with or organizing alongside community groups. This echoes findings from studies for researchers to build personal relationships with the communities they work with (Sum et al. 2025; Harrington, Erete, and Piper 2019). Forefronting trust and building strong relationships in recruitment also helps tackle a core question – who is actually participating in Participatory AI – by encouraging participation from marginalized groups.

There are existing mechanisms for researchers to join community-building efforts. Interviewees mentioned building spaces for community groups and organizations to create relationships with AI practitioners, such as at the FAccT conference, where “if you make space for more and more and more community-based organizations and advocacy groups to present there, you do build real relationships” (P4). Another interviewee (P2) mentioned creating academic spaces where a principal investigator can maintain many relationships with community organizations to support PhD projects, and summer programs where a senior academic matches undergraduate and masters students to community groups. P3 discussed creating a formal network of community organizations and advocacy groups with similar concerns: “the project is more well known and global … so sometimes people find us.” Maintaining such a network included holding regular events for interested groups and extending regular invitations to engage with the researchers, such as relevant research colloquiums and symposiums, resulting in long-term connections and collaborations over time (P3, P4). Although labor-intensive, building spaces that can help connect expertise, potential funding, and need is a promising approach to growing personal and professional networks without an individual researcher or community needing to do it all themselves.

Once researchers are able to establish themselves within a network, it is far more likely that community organizations will approach a researcher with a proposal. One interviewee said, after participating in a project with an organization that was organizing similar community groups, “this relationship unlocked all these relationships to local community-based organizations” (P4). Describing how these relationships eventually led to a collaboration, P4 described how “a contact was looking for some help from us . . . [and] as that conversation evolved . . . we realized there was a lot of overlap.” Once an interviewee had one successful collaboration, they were able to focus on recruiting similar organizations with similar structural needs that could be addressed by the algorithmic intervention that they had created (P3), showing a reputational benefit for similar communities.

Besides benefits to the researcher, earlier engagement with communities can help all participants get involved at earlier stages of projects and better engage with other stakeholders, potentially building a sense of shared ownership and algorithmic interventions with a clear and sustained social impact. Moreover, technical interventions made with engagement from the relevant communities early on are key to anticipating needed changes in the design phase, reflecting recommendations to “go out in the field” (Perrault et al. 2020). Earlier engagement also makes it easier to clearly agree on the potential benefits at the beginning of the project, using the findings in this work and others including Lin et al. (2024); Ismail et al. (2023); Erete et al. (2025) as guidance to frame such conversations. When researchers are in community with the broader, local community, there are possibilities for directly recruiting community members themselves in a grassroots manner, rather than recruiting through formal institutions that represent them.

While building community is promising to further Participatory AI goals, we acknowledge that the challenges detailed in Section 5 remain. Individually, a researcher may need to adjust expectations and narratives to make the work “legible as a research contribution to a specific community” (P3). Without changes to the institutions that support researchers and communities alike, fully overcoming many of these challenges, from funding to coordination to sustaining long-term relationships, will remain difficult. We believe that sharing challenges and successes will help improve recruitment methodology.

6.2 Reflexive Documentation

During our analysis of the corpus, we saw significant disparities in the detail of documentation of recruitment strategies. When we interviewed researchers, we learned many more details about recruitment than from the researcher’s published papers. These stories included the details and difficulties of finding funding, navigating specific social networks, engaging with specific actors to build relationships with community partners, and working with a set of institutions, such as universities and conferences, to help researchers build the relationships to recruit. As such, we recommend better documenting recruitment practices. Our recommendation to document these “backstage” aspects (Bødker and Kyng 2018) draws from a tradition of researcher reflexivity to improve transparency and create better recruitment methodologies in the field. Another benefit of introducing this reflexivity is to make the invisible work of recruitment and establishing relationships more visible to the research community, which can initiate change in how this labor is valued by institutions (Le Dantec and Fox 2015). Reflexivity is also a tool that can allow researchers to more clearly understand the complexities of power dynamics that impact their research projects, including the hierarchies, interests, and worldviews driving the decisions in research (Miceli et al. 2021). We now attend to three factors of reflexive documentation for participatory projects: social and personal networks, institutional support, and learning from refusal. These three factors are introduced as a starting point for reflexive documentation.

Personal networks

In our corpus of Participatory AI projects, how recruitment leveraged personal networks was not always very clear, yet our interviewees emphasized the importance of personal networks to their recruitment methodologies. We suggest that researchers record which relationships were key in building community partnerships, how these relationships were initially formed (as raised by P2), and how they resolved any conflicts in the relationships. Detailed narratives of recruitment are given in Le Dantec and Fox (2015); Delgado, Barocas, and Levy (2022), and Tandon et al. (2022), but even brief descriptions would help provide additional context for others recruiting.

Institutional support

As researchers navigate their own capacities for engaging in Participatory AI, guidance on finding funding is key. One interviewee in our study strongly suggested documenting how researchers recruited funding in Participatory AI projects (P2). Beyond sharing strategies for identifying such funding sources, it would be useful to share details about the structures of institutions that support Participatory AI development. As an example of this, one of our interviewees spoke to how replicating successful institutions in other universities that enable Participatory AI development could help their own university support similar endeavors to improve recruitment outcomes (P1).

Refusal and non-response

Researchers tended to document the successful recruitment strategies, but often had stories of unsuccessful recruitment strategies that were not documented. Recruitment non-response and the failure of strategies to produce successful partnerships are also part of the recruitment process, and should be documented. Narratives of these failures and how they were overcome can provide starting points for other researchers designing recruitment methodology. As P5 put it, “finding out what went wrong is almost more useful than finding out what went right, because there are certain things that…everyone would do differently in each of the papers.”

Documenting the instances of refusal to participate can provide light on which communities are underrepresented in Participatory AI and why. AI designers have called for supporting refusal in design (Barocas et al. 2020), and P4 noted that “[academic and nonprofit spaces] can be… almost violent” for certain people, highlighting the importance of refusal as a mode of participation. Using data collected about refusal, Participatory AI can create methods to incorporate refusal as an active mechanism of governance and accountability, potentially building on models of informed refusal in bioethics (Benjamin 2016).

6.3 Towards Representativeness

Finally, representativeness and meaningful participation are key goals in Participatory AI, yet our interviews highlighted the difficulty achieving these goals due to logistical challenges and challenges with cold calling potential partners, for instance. Right now, finding partners and communities is done largely ad-hoc. Even when done well, it is hard to learn what worked well, let alone to reproduce the methodology in new projects and new domains. In part, this is a problem of documentation, but it is also because researchers may not have the same access or context other researchers had. We believe that there would be value in combining relationship building with a more systematic recruitment methodology. Ideally, a systematic recruitment methodology could be utilized by many researchers to improve representativeness, even in new contexts, and could allow repeated evaluation of methodology across many projects. A focus on recruitment methodology can highlight the key role that affected communities should play in developing AI.

7 Limitations and Future Directions

The selection of the corpus and interviewees reflect our positionalities as researchers based in a research institution in the Global North, as well as our personal networks and publication goals. In addition to interviewing AI researchers, community groups that have partnered with AI researchers could provide more insights on these challenges, particularly from their perspectives and needs. Existing recruitment practices of other kinds of practitioners of Participatory AI, such as community groups that organize and gather data about AI systems, are also under-represented and deserve further study. Together, these directions can help advance the goal of improving Participatory AI and AI for Social Impact practice.

Acknowledgments

We would like to thank our interviewees for taking the time to converse and share their perspectives with us. We would also like to thank the reviewers, as well as members of the Computational Social Science group, Realize Lab, the Political Economy and Algorithms group, Nathan Kim, Shreya Chowdhary, and others at the University of Michigan, for feedback at crucial stages of the paper.

Appendix A Interview Questions

Prior to the study, we emailed the participants with the specific questions from ParticipAIte (the preliminary database we have developed, Appendix B) that we planned on asking during the session, as well as the consent form. The interview was a semi-structured interview, which entailed follow-up questions on a case-by-case basis. See the questions outlined below.

• •

Introduce the project: “Hi, my name is [redacted]. Thanks for being here. First, I want to check if you’ve read the consent form, and I can answer any questions you might have about what we’re doing today. I also want to double check that you are consenting to the study protocol we have planned today, including recording.” “Is it OK for me to hit the record button now?” “For context, we’re working on a project to develop methodology for Participatory AI. Participatory AI aims to develop AI with interested and impacted communities, and ultimately center their needs and values. We appreciated your work in this space, and would love to learn more. In particular, we are working on a project to help AI practitioners standardize the documentation of participatory processes, like recruitment, so we can distill best practices. I will ask some questions about your work on [specific project], and we’ll discuss how it’s presented in our tool called ParticipAIte. Do you think you could talk about this project?” [Introduce the project]

• •

Next, the participant will view an entry in ParticipAIte that is filled out with details of their project, which will be completed by the authors according to publicly available scientific literature. See attached image for what an entry in ParticipAIte will look like. “This is how your project is presented in our database right now. From your paper, we were able to fill out these details. However, we still want to learn a few more details about the project.” We will ask the relevant questions from the ParticipAIte database that we want more details on. Researchers will take note of issues in understanding the content of the database during this step. After completing the documentation, we will ask some questions about their experience.

• •

“Do you think this kind of information in ParticipAIte is helpful for future research (both your research or others’ research) in this area? Why or why not?”

• •

“Why did you choose to reach out to potential participants for your project?”

• •

“Why did you choose to reach out to participants in this way?”

• •

“Were there any unexpected barriers to doing so, or challenges during the process?”

• •

“What information did you use to develop your recruitment strategy?”

• •

“What did your participants contribute to the project?”

• •

“How did your participants and involved communities benefit from the project?”

• •

“Did you feel that all relevant stakeholders were included, particularly those from traditionally marginalized groups? Why or why not?”

• •

Next, we look through the database together, focusing on one existing case study that is recorded from the AI literature. “Next, I have a different case study I want to show you. Let’s take a look at this together.”

• •

“What are the differences you notice between this case study and yours?”

• •

“Can you tell me more about the difference in recruitment strategies that you notice?”

• •

“Is there anything you would have done differently in your project that we just discussed, now that you’ve seen the case study/entry?”

• •

Finally, we ask for some broad feedback on the tool: “Do you have any suggestions about how we should design the database to help AI practitioners develop strategies for a truly participatory AI?”

Appendix B ParticipAIte, A Database for Community Documentation of Recruitment Strategies

One of our motivations and beliefs in writing this paper is that specific documentation can encourage transparency and reproducibility for future projects. As a first step, we created a preliminary database called ParticipAIte, that we hope could inspire further transparency and reproducibility in the future.

When adding a new case study to ParticipAIte, a form is presented to a user, who we will refer to as a scribe. First, the scribe is asked to identify their role in the project, and list the distinct stakeholder groups involved in the project. Then, for each of the stakeholders, scribes will specify what kind of roles the stakeholder played in the project, who initiated the project, the recruitment strategy that was used to recruit these stakeholders, and any important lessons that were learned from this process of recruitment.

We now describe each section of the form in detail.

• •

List the types of stakeholders involved in the project and their role(s) These roles are synthesized from codes from our corpus identified in Section 3: Clients, Collaborators, Decision Users, Data Creators, Decision Subjects, and Regulators and Governors.

The roles are included to help those filling out the database envision the different groups of people that may have participated in the development of the AI system, whether they were construed as research subjects and formal participants or not. As noted in our coding of “Who were the different stakeholders involved in the project?”, we note that stakeholders can fall under multiple categories, and researchers can engage with more than one stakeholder groups and use different recruitment strategies with each group. This is not necessarily an exhaustive list of stakeholders.

• •

For each stakeholder, who initiated the recruitment process? We recall that recruitment can be initiated by researchers or participants, and leave space in this form for indicating who initiated the process of recruitment.

The next question is:

• •

What mechanisms did you use to recruit your stakeholder? The scribe can select an answer from the following four choices, which will lead to additional subquestions that are included for specificity.

We contacted and worked with an NGO, university, advocacy group, union, etc. to recruit these stakeholders. This corresponds with the “Organizational” code.

• •

If appropriate to disclose: Which Organization?

How did you know about this organization? 2. 2.

I advertised the opportunity directly on a mailing list, crowdworker platform, on the wall or train, on a website, etc. This corresponds to the “Infrastructural” code.

• •

On what mode of communication did you advertise? 3. 3.

I leveraged existing personal networks as a recruiting strategy. This corresponds with the “Personal Networks” code.

• •

Describe how the personal network was used in recruiting and what the personal connection was. 4. 4.

I met the participant at an event, such as a conference, a civic advocacy meeting, etc. This corresponds to the “Event” code.

• •

Which event?

• •

What important lessons did you learn from recruiting this stakeholder? This question is added to encourage reflection on the recruitment process and to share any special learnings about the recruitment strategy.

• •

What did the call for participation look like? This question is included to improve transparency of the recruitment process and provide concrete examples of how to design a call for participation for similar case studies.

Appendix C Tables

We next include tables with the results of coding our corpus (Tables 4, 6, 7, 6, 8, 9). These results inspired the questions included in ParticipAIte.

Appendix D Corpus Tables

We also present statistics related to the corpus. Table 10 sorts projects in our corpus by represented countries. Table 11 sorts projects by domains. Table 12 sorts projects by the participation entry point defined by (Corbett, Denton, and Erete 2023).

Appendix E Corpus Links

Finally, we include the 37 case studies in our corpus in table format with links. Links are verified at the time of publication in 2025. If copying a link from a PDF, please be sure to include underscores (_).