Use of the Delphi method as an instrument of community participation in health needs assessment

TL;DR

This paper shows how the Delphi method helped communities in Spain participate in identifying local health needs and influenced policy decisions.

Contribution

The study demonstrates the Delphi method's effectiveness in community health assessments and policy development in a local government context.

Findings

The Delphi method enabled successful community participation in health needs assessments.

Prioritized health items were incorporated into local policies, accelerating decision-making.

The approach was found to be replicable and cost-effective for local governments.

Abstract

We conducted a comprehensive analysis of the usefulness of the Delphi technique for facilitating community participation in local health needs assessments within the Andalusian Local Health Action Network, Spain. We developed an ad hoc online questionnaire based on the Social Determinants of Health model and applied it to a panel of experts in two municipalities in the province of Seville (Andalusia, Spain) between May and June 2021. Our results reflected good panelist participation. The questionnaire successfully enabled the prioritization of both new and original items, some of which were incorporated into local health policies. We concluded that the Delphi method was effective for facilitating participation in local health needs assessments offering a replicable, cost-effective approach that accelerated local policy development and supported the implementation of Health in All Policies within local government.

Full text

Key messages

• The Delphi Method (DM) is a cost-effective method for assessing and prioritizing health needs with community participation.
• The DM helps to define local health policies in dispersed or under-resourced areas.
• A Social Determinants questionnaire facilitates local implementation of Health in All Policies.

Introduction

Since the release of the Ottawa Charter for Health Promotion in 1986, substantial evidence has emerged highlighting the critical role of social determinants in influencing health outcomes [1, 2]. As a result, intersectoral action has been increasingly promoted to improve the quality of life of the general population [3]. International [4], European [5], and Spanish [6] institutions have endorsed the Health in All Policies (HiAP) approach [4], which requires policymakers in all sectors to systematically consider the health impacts of their decisions [7]. Local governments play a crucial role in this framework, as they are the level of governance closest to the population and can implement policies that more accurately reflect local needs [8]. The development of HiAP at the local level is often based on health needs assessments (HNA), which rely on community participation [8] to ensure that policies resonate with citizens [10] and to design interventions aimed at improving health [9].

In our analysis, we explored the use of the Delphi method to facilitate community involvement in the HNA process. This initiative was conducted within the framework of the Local Health Action Network (RELAS), a project sponsored by the Andalusian Ministry of Health and Consumer Affairs to promote the implementation of HiAP by developing local health plans (LHPs) based on health needs assessment data [9], consisting of both quantitative data (demographic, epidemiological, economic, geographical, and social indicators) and qualitative insights gathered from the community through participatory methods such as focus groups, surveys, and interviews. A steering group then analyses this information to prioritize health needs and guide the development of interventions [9].

In the RELAS framework, this steering group, known as the Technical Advisory Group [11], comprises members from various health administration areas, with leadership provided by a provincial-level Expert Technical Advisor in health promotion. The Advisor supports and guides municipalities throughout the process.

During the COVID-19 pandemic in spring 2021, traditional participatory tools were impractical due to the need for face-to-face interaction. To prevent disruption of the process and ensure community input, the Health Promotion Team for the Aljarafe-North Seville Health District—composed of the Expert Technical Advisor, the Director of Disease Prevention, Health Promotion and Surveillance Unit, and leaders from the Local Health Action Network—opted to use the Delphi method.

We sought to analyze the implementation process and results of our experience with the Delphi methodology to promote intersectoral participation in the development of local health policies in two municipalities in the province of Seville, Andalusia.

Data and methods

The Delphi technique is widely used in health sciences research to identify priorities and reach group consensus on key issues [12]. It is a structured method designed to gather expert opinions on a given topic [13]. In our analysis, we replicated key features of the Delphi method: purposive sampling, participant anonymity, iteration, and controlled feedback [14].

Fieldwork was conducted between May and June 2021 in two municipalities within the Aljarafe-North Seville Health District (Mairena del Alcor and Mairena del Aljarafe). After failing to find an existing instrument suited to our objectives, we designed an ad hoc online questionnaire based on Dahlgren and Whitehead’s (1991) framework of health determinants [14]. This framework is widely used due to its simplicity and practicality. The questionnaire, reviewed by experts and public representatives, was approved with only minor revisions.

The final version included 48 closed questions across six components: habits and lifestyle; education system; social environment; socioeconomic conditions; physical environment and neighborhood; and healthcare system. Participants rated the importance of each item on a 5-point Likert scale (from 0 = least important, to 5 = most important) and had the option to suggest new items relevant to their municipality.

In Rounds 1 and 2, participants rated the same items, along with new ones suggested after Round 1. The importance ratings from Round 1 were provided for reference, and the same criteria were used for re-rating. Each round lasted two weeks, with email reminders sent to non-respondents to ensure participation.

The Expert Technical Advisor processed the data and prepared a report on the results. The priorities identified through the Delphi process were added to the HNA and included in the local health plan, adopted by local councils to support the HiAP strategy.

The Health Promotion Team defined the selection criterion for experts as membership of the community, considering individuals who lived or worked in the municipality (geographical criterion) or belonged to a local sector or group (sector criterion). The selected expert profiles included representatives from citizens’ associations, key informants familiar with the municipality (for example, representatives of political groups, trade unions, and employers’ associations), municipal technical staff (from departments such as housing, environment, and public safety), staff from public institutions (schools, social services), primary care professionals (doctors, nurses, social workers), community pharmacists, and representatives of vulnerable groups.

The Health Promotion Team contacted potential participants by phone to invite them, explain the purpose of the analysis, and introduce the Delphi methodology. Individuals who demonstrated willingness, commitment, time availability, and communication skills [15] were included. Those who agreed to participate received an email from the Expert Technical Advisor with instructions, a link to the questionnaire, and the response deadline.

Data analysis

We defined consensus in advance [16] and established two criteria: consistency of responses between rounds and the absence of new item contributions. To check consistency, we applied the Kruskal–Wallis test, comparing median scores for each item across rounds, with a significance level of p ≤ 0.05. To prioritize items, we used two conditions: a median score of 4 or 5 (indicating high importance) and an acceptable consensus level (relative interquartile range (RIR) ≤ 75%) [17]. Data from the online questionnaire were transferred to Open Office spreadsheets, and statistical analyses were performed using Epi Info 7 software [18].

Results

Since our main objective was to analyze the usefulness of the Delphi method for facilitating community participation in local HNAs, we emphasize the application of the methodology. We also provide a brief analysis of the prioritized items as they relate to facilitating the development of local health policy.

There was a high level of participation relative to the initial number of people invited. The attrition rate between rounds was higher among male participants (Table 1). The distribution of items per health determinant varies by municipality, reflecting the addition of new items that correspond to specific local characteristics (Table 2). The distribution of prioritized items corresponds to the percentage of those included in the original questionnaire and those proposed by panelists (Table 3).Table 1. Participant flows by sex and municipalityMairena del AlcorMairena del AljarafeRound 1Round 2Round 1Round 2Number invited46 (100%)35 (100%)45 (100%)28 (100%)Number of participants35 (76.0%)25 (71.4%)28 (62.2%)26 (92.8%)Sex Female31 (68.0%)18 (70.0%)26 (57.8%)19 (73.3%) Male15 (32.0%)7 (30.0%)19 (42.2%)7 (26.7%)Table 2. Evolution of items addressed in two Delphi roundsMairena del AlcorMairena del AljarafeItems grouped by SDH themeIncluded in questionnaireProposed by panelistsNo. prioritized of those included in the questionnaireNo. prioritized of those proposedIncluded in questionnaireProposed by panelistsNo. prioritized of those included in the questionnaireNo. prioritized of those proposedHabits and lifestyles72307451Education system98009600Social environment72127600Socioeconomic conditions63516361Physical environment and neighborhood13300131503Health system63226626Total482111548401311Table 3Items included and prioritized after two Delphi roundsItems Round 1Items Round 2No. included in questionnaireNo. proposed by panelistsNo. included in questionnaireNo. proposed by panelistsNo. prioritized of those included in the questionnaireNo. prioritized of those proposedMairena del Alcor482169011* (22.9%)5* (23.8%)Mairena del Aljarafe484088013* (27.1%)11* (27.5%)*Only prioritized in Round 2

The steering group included some of the prioritized items (4 in Mairena del Alcor, 5 in Mairena del Aljarafe) in the LHPs, which the municipal councils then approved. Three prioritized problems associated with habits and lifestyle (poor diet, sedentary lifestyle, stress) were common to both municipalities. In the case of Mairena del Aljarafe, problems related to the health system were excluded from the LHPs and were referred to the management team of the reference Health District.

Discussion

Our analysis aimed to examine the effectiveness of the Delphi method in promoting participation in Health Needs Assessments at the local level. It addresses a gap in the literature concerning the lack of concrete experiences with Health in All Policies, especially regarding implementation and practical recommendations [8, 19, 20], which limits local policy development within this paradigm [8]. The literature also emphasizes the need for research on facilitators and barriers to HiAP strategy development [8], particularly in the participation phase [19], a crucial element that influences policy decisions in the HiAP process [8, 20]. While focus groups and key informant interviews are commonly reported techniques, the use of the Delphi method for HNA is relatively new [9].

We found that the Delphi methodology was an efficient way to obtain a consensus list of SDHs in both municipalities. The literature supports the advantages of gathering information through this internet-based tool compared to face-to-face participatory techniques, highlighting its low cost and savings on typical expenses like participant travel and venue hire [21, 22]. Ravaghi et al. (2023) identified several challenges in conducting an HNA that Delphi can address [9]. First, Delphi is replicable and can engage the community without sacrificing methodological validity [23]. Second, the use of new technologies reduces logistical time and saves human and economic resources [16, 24]. We identified a potential barrier in the HNA process: local health professionals may lack skills in building consensus through participatory decision-making. To mitigate this, an external agent could administer the questionnaire to participants. Some authors have noted challenges in ensuring adequate participation and the risk of legitimizing pre-made decisions without allowing diverse citizen views to emerge [10]. The Delphi method can resolve this by facilitating participatory dialog over several rounds and allowing panelists to add their own opinions through open-ended questions.

In the RELAS project, we identified the following advantages. First, the Delphi method accelerates the participation phase compared to other methods, when conducting an HNA takes so long that it delays the preparation of the LHP [23]. Using this method, we reduced the overall process from an average of three years to one year. Second, with an external technical advisor facilitating the process, a single person can apply the tool in multiple communities. Third, it is possible to increase the number of panelists without a corresponding increase in complexity. Finally, the replicability of the technique allows us to identify changing needs within the same community and to compare different communities while identifying common factors influencing local needs. This method could play an important role in health areas where communities are scattered, where it is difficult for key informants to travel, or where the cost of human and economic resources is prohibitive.

We believe that community HNAs should focus on the social determinants of health [9]. Accordingly, we designed our initial questionnaire around Dahlgren and Whitehead’s model, which has been successfully used by practitioners and policymakers outside the health sector to conceptualize determinants [14]. Our questionnaire included a specific list of items for each determinant to encourage panelists to consider the broader context of social determinants of health when responding and to stimulate the suggestion of new items for each determinant. While the length of the questionnaire may contribute to participant attrition [25], there are no explicit criteria for defining an optimal number of items to prevent this. Following Lilly et al.’s recommendation, we grouped items into blocks [21]—in our case, by type of determinant—to help panelists maintain a determinants-based perspective of local reality and propose new items for each block [14].

We acknowledge that the criterion for inclusion as an expert panelist is controversial in the literature due to the challenge of measuring expertise. Therefore, it is necessary to define this criterion before forming the panel [13, 26]. Following this recommendation, we established community membership [9] as our criterion, using geography as the primary reference and representation of various sectors and local groups (health, education, the elderly, children, equality, etc.) as secondary. We considered panel members to be experts because they live or work in the municipalities and possess first-hand knowledge of the local realities. In the scientific literature, such individuals are referred to as affected parties, involved parties, or key informants in diagnostic processes who have knowledge of the situation under analysis regardless of title, status, or rank. They contrast with experts, who must have outstanding qualifications in the subject under study, such as academic credentials, special qualifications, or notable professional experience [27]. We set an initial minimum of 40 panelists, anticipating attrition in subsequent rounds, and thus ensured a diverse range of participant views in both municipalities. While there is no consensus on the optimal number of panelists, between 10 and 30 is generally considered sufficient [13, 20, 26].

After analyzing the prioritization results, we noted that the same three items related to lifestyle habits were prioritized and included in the LHPs of both municipalities [26]. This may be because these determinants are obvious and well known, suggesting there is still room to improve the dissemination of the Health in All Policies approach. Contributions made during the Delphi process could help the steering group identify new local realities requiring intervention.

The literature suggests that translating Health Needs Assessment findings into specific policies has been very limited. Ravaghi et al. (2023) conducted a scoping review of 169 studies: only five mentioned successful collaboration between the health sector and policymakers, and only four succeeded in setting or changing policy agendas [9], highlighting the lack of linkages between proposals resulting from participation and health policies [10]. The literature also mentions that political representatives often do not trust the results of citizen participation focused on social determinants of health [28]. In their 2023 review, Ling et al. discussed factors influencing the HiAP process in local governments and note the ambiguity and diversity of aspects affecting political decision-making, which may be one reason why some issues prioritized by citizens are not included in LHPs [28]. This was the case with items related to health services in Mairena del Aljarafe, which were excluded entirely.

Study limitations

We identified most of the limitations in our analysis as relating to the panelists. First, because we did not stipulate a minimum number of representatives per sector but rather a total number, some sectors may have been over- or underrepresented, potentially affecting the number of prioritized items per determinant (more health sector panelists prioritizing more health-related issues). Second, we were unable to verify the level of expertise of the panelists. Since it is difficult to quantify community membership, we could not determine whether individuals living or working in the community were knowledgeable about it, especially if they had recently moved there [16]. Third, ensuring the independence of the panelists was challenging, as participation was limited to those with a pre-existing interest in the topic or a relationship with the project [25]. This relates to the difficulty of controlling for bias from panelists with personal or sectoral views of local realities [9]. Fourth, we did not directly check that the panelists understood the items included in the questionnaires. Differences in interpretation may limit consensus using this technique [16]. Conducting an online session via videoconference beforehand to explain the project and procedures could mitigate this limitation. Fifth, although we included local stakeholders, leaders, and key figures with a broad vision of various needs and social groups, we did not consider including individuals who lacked Internet access, had limited literacy, or belonged to particularly important disadvantaged groups in the locality. Furthermore, the initial validation of the questionnaire was a very simplified process due to the lack of a similar, ready-made tool. As a result, the questionnaire may have included items that were too general or open to different interpretations. We do not know whether the list of items prioritized by the panels would have been different if other techniques had been employed. Furthermore, we were unable to compare our results, even partially by sector, because there are no other similar local publications on these topics.

A common limitation of participatory processes is the ethical challenge of managing participants’ expectations about the ultimate impact of their contributions [9]. Although we made it clear throughout that Delphi prioritization was part of a broader process, we did not evaluate panelists’ perceptions of the process or the results.

Future directions

With this analysis, we have paved the way for further development in designing and validating an initial questionnaire model that can be used to conduct Health Needs Assessments with a focus on Social Determinants of Health in local settings. Based on a simple, concrete, and accredited methodology, this tool would provide local governments with the opportunity to implement Health in All Policies and to assess whether channeling citizen participation in the HNA process through a specific validated tool enables policymakers to feel more comfortable implementing HiAP at the local level.

Another avenue for future research with the same objective is the evaluation and validation of panels specifically for the use of the Delphi method in local settings. While there are recommendations in the literature for creating and energizing expert panels, most are at the scientific level, typically focusing on assessing expertise on a single topic to be analyzed. Expert assessment may mean expertise in the local area by virtue of living and working there (geographic criterion) and expertise in a specific determinant in that locality (sector criterion). Therefore, it is crucial to establish specific criteria for panel composition and to assess the extent to which uneven representation of different sectors within a panel influences the results obtained.

Another future line is to analyze experts’ expectations and satisfaction with the method and the tool, as this may help identify possible biases and causes of attrition during the process.

Conclusions

In this Viewpoint, we presented the mechanism and process used to produce a health needs assessment and its degree of implementation in a local health plan approved by local government, but the medium- to long-term development and impact of this policy remain to be evaluated. We consider the Delphi method to be an effective and cost-efficient strategy for involving citizens in the development of local health policies, particularly in highly geographically dispersed areas or when accessibility is limited, such as during confinement phases of the COVID-19 pandemic.