Systematic Evaluation of How Indicators of Inequity and Disadvantage Are Measured and Reported in Population Health Evidence Syntheses

TL;DR

This study examines how population health reviews consider inequity and disadvantage, finding limited focus and data for subgroup analyses.

Contribution

The study systematically evaluates the use of equity indicators in Cochrane reviews and highlights gaps in subgroup analysis completion.

Findings

Only 22.9% of reviews focused on wider health determinants, with most on lifestyle or healthcare interventions.

Fewer than 6% of reviews explicitly used PROGRESS-Plus indicators for equity analysis.

Age was the most common subgroup indicator, but many planned analyses were not completed due to insufficient data.

Abstract

We present a systematic evaluation of population health reviews from the Cochrane Database (January 2013–February 2023) to evaluate how indicators of inequity or disadvantage are considered and reported in population health evidence syntheses. Descriptive analyses explored a representation of reviews across health-determinant categories (primary and secondary categories), summarised equity-focused reviews, and examined proportions and types of reviews that planned/completed a subgroup analysis using ≥1 indicators from the PROGRESS-Plus framework. Of 363 reviews included, a minority focused on interventions targeting wider determinants of health (n = 83, 22.9% as primary category), with a predominance related to individual lifestyle factors (n = 155, 42.7%) or health care services intervention (n = 97, 26.7%). An explicit equity focus was evident in 21 (5.8%) reviews that used PROGRESS/PROGRESS-Plus, and 28 (7.7%) targeting vulnerable groups. Almost half (n = 165, 45.6%) planned a subgroup analysis by ≥1 PROGRESS-Plus indicator, which was completed in 63 reviews (38.2% of 165). The non-completion of planned subgroup analyses was attributed to insufficient data (too few primary studies, data not reported by subgroups). Among the 165 reviews planning a subgroup analysis, age was the most cited indicator (n = 91, 55.2%), followed by gender/sex (n = 67, 40.6%), place (n = 47, 28.5%) and socio-economic status (n = 37, 22.4%). This study highlighted missed opportunities for learning about the impacts of health equity in population health evidence syntheses due to insufficient data. We recommend routine use of PROGRESS-Plus and greater consistency in socio-economic proxies (occupation, education, income, disadvantage measures) to facilitate meta-analyses and subgroup analyses.

Figures

PRISMA flow diagram of study selection (adapted from [39]).

Number of reviews mapped to Dahlgren and Whitehead determinants of health categories (numbers reflect alignment as primary or secondary categories; therefore, the total number (n = 4354) exceeds the number of reviews included (n = 363); see Supplementary File S7.

Full text

1. Introduction

Understanding and addressing the differential distribution of health outcomes and intervention effects across population groups are a public health priority [1], recognised by public health researchers, practitioners and policy makers [2,3].

The consideration of health inequity is central to the concept of population health, notwithstanding the debate regarding definitions [4,5]. Population health is determined by the wider environmental context, and different population groups vary in exposure and vulnerability to health risks, thus creating inequities in health [6]. Population health interventions can, therefore, tackle the wider determinants, or target vulnerable populations (e.g., low-income groups, less educated, homeless) who have a greater risk of poor health given their physical, economic, and social circumstance [1,7]. This aligns with Marmot’s principle of proportionate universalism, of reducing health inequity through providing support proportionate to need, whilst also addressing the wider determinants [8].

Health inequalities are the observed differences in outcomes across individuals and groups defined on the basis of socio-economic factors (e.g., income, education), geography, individual characteristics (e.g., age, sex, ethnicity), and other social factors (e.g., homelessness [9]). These also serve as an indirect means of evaluating health inequity or disparity [10]: when differences in health are considered unfair and avoidable because they result from some kind of injustice [11]. Some health differences are unavoidable, such as those linked to genetic predisposition or age [2,9]. But public health policy to reduce population health differences are contingent on them being avoidable (inequitable). Therefore, consistent with a public health remit of reducing avoidable differences within the population’s health, and with terminology used in measurement frameworks [12,13] and guidance [2,12,14], this study considers the measurement and reporting of equity/inequity.

Evidence syntheses are the primary sources of evidence used to inform public health practice and policy [15]. By combining data from large numbers of subgroups across diverse populations and settings, such syntheses should facilitate explorations of equity [16]. Yet, meta-analyses are often undermined by heterogeneity of indicators to examine equity [17,18]. The last 20 years have seen a concerted effort in this area [19]. A 2003 framework of (in)equity indicators, PROGRESS [20], was later augmented to PROGRESS-Plus: Place of residence, Race/ethnicity, Occupation, Gender/sex, Religion, Education, Socioeconomic status, Social capital, and personal characteristics [21]. Subsequent guidance for systematic reviewers is offered in the 2012 PRISMA-Equity (or PRISMA-E) extension [12,14] and Cochrane handbook [2]. Yet there is scope for varied practice: which population characteristic to use and the potential conceptual overlap between them (e.g., place can be a proxy for various area-level characteristics), which indicators best represent a given characteristic (e.g., educational attainment or years completed) and how to categorise them (e.g., low/high; primary/secondary/tertiary education). The relevance and measurement of characteristics might also vary with geographical, social or cultural context (e.g., low–middle- vs. high-income countries).

Others examining equity reporting in evidence syntheses have confirmed inconsistency in how inequality/inequity is reported in health inequality/inequity-focused reviews. Hollands et al. [22] found many reviews that used or intended to use PROGRESS-Plus (through targeted searches). Others have observed varied use of PROGRESS/PROGRESS/Plus and PRISMA-E, and variation in the extent of subgroup analyses [23] and how checklists could or should be applied [22]. Welch et al. [24] reviewed 158 methodological studies that examined how systematic reviews assessed health equity. Most used a descriptive assessment of equity reporting and analysis (140, 88.6%), with 58 studies assessing whether reviewers conducted subgroup analysis using one or more PROGRESS-Plus characteristics.

The present study builds on these reviews, evaluating population health evidence syntheses, where health equity should be a routine consideration. Our aims were to: (i) explore the representation of population health reviews across health-determinant categories; (ii) describe those with an equity focus; (iii) examine the proportions and types of reviews that planned/completed subgroup analyses using ≥1 indicators from the PROGRESS-Plus framework.

2. Materials and Methods

2.1. Design and Search

This was a systematic evaluation of a cohort of Cochrane reviews of population health research [25]. It took the form of an overview of reviews [26] following a modified PRISMA reporting format (Supplementary File S1). The data source was the complete Cochrane Database of Systematic Reviews from 1 January 2013 to 19 February 2023 (n = 5953). The full protocol is available at Research Registry (reviewregistry1717).

2.2. Inclusion and Exclusion Criteria

Inclusion/exclusion criteria were refined during an initial calibration phase. We aimed to capture evidence syntheses relevant to population health, with potential for sub-analysis (planned and/or completed), or that targeted ‘vulnerable’ populations. This was to ensure that reviews not specifying an equity focus (a noted weakness [12]), but that could or should have, were not missed. Potentially eligible types of syntheses included reviews of non-clinical interventions (e.g., population health, behavioural and educational interventions, mass media campaigns); reviews of reviews; rapid reviews of population health interventions; reviews of prognostic and non-clinical prototype public health studies.

Reasons for exclusion are detailed in Supplementary File S2. Briefly, we excluded reviews of: diagnostic accuracy studies; qualitative studies; clinical intervention studies; clinical populations; other specific populations (e.g., women with multiple births, people undertaking cosmetic procedures); only low- and middle-income countries (LMIC) to avoid additional complexity through differences in the wider economic, geographical and health context between high-income countries (HIC) and LMICs [2], and related differences in equity measures [27]; health service design, organisation of care and healthcare professionals’ practice; and individual-focused intervention studies. We also excluded scoping reviews as they did not include quantitative syntheses with potential for subgroup analyses, and methodological reviews focused on the methods or processes of research (e.g., participant recruitment, retention or randomisation, or different statistical methods).

2.3. Screening

Following a calibration phase using 180 consecutive titles and abstracts (by CG/AR), finalised criteria were used to screen titles/abstracts of 5953 Cochrane Reviews (by CG), with independent assessment (by AM) of a 10% sample selected using an MS Excel random number formula (target of 90% agreement). Full texts of potentially eligible reviews were screened, with independent verification (by AM, 10% random sample, target of 90% agreement). Disagreements were resolved through discussion.

2.4. Outcome Selection

Key outcomes included: the mention of inequity, inequality or social patterning in the introduction/methods; the explicit use of PROGRESS/PROGRESS-Plus checklists; targeting a vulnerable population group; explicit planning and/or completing subgroup analyses using PROGRESS-Plus indicator types; indicators used; how they were categorised.

2.5. Data Extraction and Analysis

All data were extracted by CG using an MS Excel data extraction form (Supplementary File S3) with independent verification by AM (10% random sample). All disagreements were resolved through discussion. Reviews were not quality appraised given the lack of relevant appraisal tools and limited guidance available for methodological reviews [28]. The analysis was descriptive: mapping reviews to the appropriate Dahlgren and Whitehead health-determinant categories [29], assigning primary and secondary categories if more than one was applicable; describing if/how equity was examined (Supplementary File S4).

3. Results

3.1. Results of Screening

The screening of 5953 review title/abstracts identified 396 full texts that were assessed for eligibility (Figure 1). Thirty-three full texts were excluded, leaving a final sample of 363 reviews (Supplementary File S5). The most common reasons for exclusion were study populations having existing conditions (n = 12) or including only studies from LMIC countries (n = 11) (Supplementary File S6).

3.2. Characteristics of Reviews

When mapped to the determinants of health categories, the largest proportions of reviews mapped to individual lifestyle factors (n = 155) or health care services (n = 97) as a primary category, followed by education (n = 30), other (n = 24) and work environment (n = 22). Less than one-quarter of reviews aligned with wider determinants (n = 83 as primary category), reflecting physical living environment (e.g., housing, water sanitation) and broader socio-economic conditions (e.g., general socio-economic conditions, unemployment) or the social environment (social and community networks) (Figure 2; Supplementary File S7).

3.3. Measurement and Reporting of Health Inequities or Inequalities

Half the reviews referred to inequalities, inequities or social patterning in the introduction/methods (n = 181, 49.9%). Twenty-eight (7.7%) focused on interventions targeting vulnerable populations, most commonly people with experience of abuse (n = 6), caregivers (n = 4) or people working in environments that expose them to risk (n = 3) (Table 1).

Twenty-one (5.8%) of the 363 reviews used PROGRESS [n = 7, 1.9%] [30,31,32,33,34,35,36] or PROGRESS-Plus [n = 14, 3.9%] [37,38,39,40,41,42,43,44,45,46,47,48,49,50] to extract data and consider equity impacts and disadvantage (Table 2). All were published after the 2012 PRISMA-E extension was introduced [12].

PRISMA flow diagram of study selection (adapted from [39]).

Number of reviews mapped to Dahlgren and Whitehead determinants of health categories (numbers reflect alignment as primary or secondary categories; therefore, the total number (n = 4354) exceeds the number of reviews included (n = 363); see Supplementary File S7.

3.4. Use of PROGRESS/PROGRESS-Plus

Table 2 summarises the 21 reviews that used PROGRESS or PROGRESS-Plus. All mentioned inequity/inequalities in the introduction/methods, and one focused on a vulnerable group (young people experiencing homelessness [40]). Two reviews did not plan a subgroup analysis by any PROGRESS indicators despite the equity focus [38,46]. A review of psychosocial support for smoking cessation in pregnancy completed most analyses for three PROGRESS-Plus indicators, with a narrative synthesis for others [39].

Eleven reviews reported a narrative synthesis around equity, some with accompanying tables or appendices detailing equity considerations in each study. Of these, six also completed subgroup analyses for some/all of the intended indicators [30,34,36,47,49,50], whereas five planned but did not complete analysis [32,40,42,48,51]. Five reviews included less detailed narratives around equity. Key points were covered, but data were not presented by PROGRESS indicators, often due to a lack of information in primary studies [35,37,41,43,45]. Two reviews, both focused on taxation to improve the health content of food, offered limited consideration of equity impacts [31,33].

Of the 21 reviews that used PROGRESS/PROGRESS-Plus, the predominant determinant of health categories were agriculture and food production (n = 9) and individual lifestyle factors (n = 9) (Table 2).

3.5. Subgroup Analysis by PROGRESS-Plus Indicators

Less than half of the 363 reviews (n = 165, 45.7%) planned a subgroup analysis using one or more equity indicators, of which 63 (38.2% of 165, 17.4% of 363) completed analyses. Of the 181 reviews that mentioned inequality, inequity or social patterning in the introduction/methods, 110 (60.8%) planned a subgroup analysis, 71 (39.2%) did not.

Figure 3 illustrates the relative numbers of reviews aligned with each health-determinant category (brown bubbles—wider determinants; blue—other categories) and that planned subgroup analysis. Although relatively few focused on the wider health-determinants, within-category proportions that planned a sub-group analysis were highest for these categories (≥60%, work environment, unemployment, general socio-economic, agriculture and food production, living and working conditions and education). Conversely, for reviews mapped to healthcare services and individual lifestyle factors, the proportions planning a subgroup analysis were lower (≤40%), despite a greater number of reviews.

Figure 4 shows the relative numbers of reviews that planned and completed a subgroup analysis, by health-determinant category. Small numbers make the percentages less meaningful, but for categories with 10 or more reviews, the completion rates ranged from 4.3% (water and sanitation) to over 25% (living and working conditions, agriculture and food production). For the most common categories of individual lifestyle factors and healthcare service categories, there were low rates of planned (40.2% and 37.9%) and completed (20.7% and 15.5%) subgroup analyses.

One hundred of the 102 (98.0%) reviews that planned, but did not complete subgroup analyses, cited insufficient data (too few primary studies overall or that reported outcomes by subgroups, or insufficient heterogeneity); two did not specify.

The indicators most intended for/used in subgroup analyses were age, followed by gender/sex, place, socio-economic status (SES) and race/ethnicity (Table 3; Supplementary File S8). There was varied practice. Age categories varied with context and target population, including specific and broad age groups, life stage or school year/stage. Gender/sex groups included a mix of gendered labels (boys/girls, men/women, mothers/fathers) and sex at birth (male/female). Place was mostly country level income (low–middle/high income) or urban/rural location. SES included a range of indicators of disadvantage/social disadvantage/deprivation, income and generic or unspecified proxies. Race/ethnicity groups reflected foci on both ethnicity and race and often compared majority/minority groups, indigenous and non-indigenous groups or groups based on disease risk. In many cases, there was a lack of specificity. Planned and completed subgroup analyses using other equity indicators were uncommon, with few examples of the ‘Plus’ indicators being considered, aside from age (Table 3).

4. Discussion

4.1. Principal Findings

This systematic evaluation of inequity considerations in population health evidence syntheses confirmed varied practices and limitations, which ultimately limit the evidence base to address health inequity. Although half of the 363 population health reviews (49.9.%) referenced inequities, inequalities or social patterning in their background/methods, few demonstrated a specific equity focus through citing PROGRESS/PROGRESS-Plus (n = 21, 5.8%), in line with the PRISMA-E [2,12], or through targeting vulnerable populations (n = 28, 7.7%).

Compared with other methodological reviews that focused on wider determinants [52] or delimited to health-equity-focused reviews [22,23,24,53], our population health focus was more inclusive. It presents a more critical picture: just 5.8% of the 363 reviews that should or could have used PROGRESS-Plus for data extraction (as a minimum) did so. This accords with Welch et al. [24] who found that, even among methodological studies of health equity assessments in systematic reviews, just 18 out 158 (11.4%) studies explicitly cited PROGRESS-Plus. A comparison with equivalent proportions reported by Hollands et al. [22] is less useful given their purposive article selection to capture the breadth of approaches (rather than comparing proportions using different approaches). In the present study, it is possible that some review authors, particularly those examining interventions not targeting the wider determinants (77% of included reviews), might not have treated their topic as population health and, therefore, not judged equity as a necessary consideration. However, regardless of the authors’ explicit focus, we identified an implicit link to population health and, often, a missed opportunity for equity consideration. Those focused on vulnerable groups explored equity impact through studying populations at greater risk of poor health as a result of unsupportive physical, economic or social circumstances [7]. This might explain why subgroup analysis was planned in only 11 of 28 reviews (completed in just one, by age and gender [54]). There are other examples when certain subgroup analyses would be unnecessary (e.g., by gender if interventions target women; by age if interventions target specific age groups; by occupation if targeting children/older adults/unemployed). These caveats notwithstanding, 39% of reviews that mentioned inequality, inequity or social patterning in the introduction or methods, did not plan any subgroup analysis to examine equity impact. This indicates that equity consideration and reporting are often not routine practice.

We focused on use of subgroup analysis as it allows reviewers to inspect broad patterns of inequity, the limitations aside (e.g., indicators considered in isolation; inability to infer causality [23]). Almost half (45.7%) of the 363 reviews planned subgroup analyses and 17.4% were able to complete some, despite only 5.8% citing PROGRESS/PROGRESS-Plus. Authors of included reviews noted insufficient data to allow analysis through primary studies being too few or not reporting data by subgroups, although the rate of subgroup analyses by one or more PROGRESS-Plus indicator was higher than the 8% (of 262 reviews, across 58 methodological studies) observed by Welch et al. [24].

This deficit in equity reporting among primary studies was reported in a recent assessment of 200 ‘equity-relevant’ studies. Karran et al. [53] found that most studies reported age, sex/gender and education (92%, 78% and 65%, respectively), and approximately half reported other socio-economic proxies (49%), race/ethnicity (45%) and social capital (44%). Yet their analysis identified an overall inadequacy and inconsistency of equity reporting which, they concluded, would likely undermine opportunities to pool data. Indeed, they illustrated that many primary studies fell short of the recommended minimum requirements for sample description by age, gender, ethnicity and a socio-economic measure [55], similar to limitations noted elsewhere [56]. We similarly found that age and gender/sex subgroups were the most used in subgroup analysis, followed by place, race/ethnicity and SES (Table 3). The low rate of planned subgroup analysis by SES (22.4%) and completed analysis in just 4.2% was particularly striking and lower than observed elsewhere [24]. This low rate, in general and relative to other PROGRESS-Plus indicators in the present review, perhaps reflects our broader population health focus, but again shows a deficit in practice.

The relative sparsity of reviews relating to the wider determinants of health speaks to a continued need for further studies and reviews, particularly given that upstream conditions are regarded the key drivers of health and equity [1]. Within this sample of 363 population health evidence syntheses, few aligned with the wider determinants, but were considered ‘population health’ given the potential to affect the social environment (e.g., social support, group-based programmes, family programmes). This under-representation was similar to that observed in a rapid review of population health reviews by Retzer et al. [57]. It reflects a general and longstanding deficit in evidence of how interventions on social determinants impact health inequity [52]. Practical challenges to developing this evidence include the feasibility of intervening in wider environmental conditions. This might require substantial investment (e.g., physical infrastructure), political buy-in (e.g., policy change) or take many years (e.g., housing development), in addition to the complexities of measuring the health effects.

4.2. Strengths and Limitations

Strengths. The strengths of this systematic evaluation included a population health focus that considered a range of health-determinant categories. The use of the Cochrane Review library as a pragmatic approach offered some advantages regarding quality assurance [58,59] and consistency in the reporting of subgroup analyses.

Limitations of the evidence included. This study highlighted the limited reporting and consideration of equity in intervention effects. As few reviews used PROGRESS-Plus or completed a subgroup analysis, there were limited opportunities for comparisons within groups (e.g., by health-determinant category).

Limitations to the processes. The single Cochrane data source did not represent the breadth of evidence syntheses. It was not feasible to complete screening and data extraction processes in duplicate for all reviews, and formal quality appraisal was not conducted, a common challenge in methodological reviews [24,28,52]. It was also beyond the scope and resources of this study to consider the full range of approaches to equity assessment [24]. Focusing on use of PROGRESS-Plus, vulnerable populations and subgroup analysis was a feasible approach.

4.3. Implications

A more consistent use of tools and processes (PROGRESS-Plus, PRISMA-E) and transparency about which equity indicators were/were not used and why would improve the standards in reporting and subsequent understanding of health inequity. This responsibility falls on both reviewers and primary study authors, for whom PROGRESS-Plus provides a useful framework, in addition to guidance from numerous related articles and reviews published in the last 1–2 years [19,22,23,24,53,55,60,61]. In particular, primary studies in population health should engage with the relevant guidance for collecting and reporting equity data (e.g., [2,60,61,62]), which would give those synthesising evidence options for analysis by population subgroups. Standardisation in approach is not appropriate given the diversity in topics, populations and context. Yet, failure to improve practice to recognise specific groups, as observed with the historical under-representation of women or aggregation of minority ethnic groups in research [63,64], perpetuates invisibility and exclusion from the evidence, leading to evidence-based practice and policy not informed by their interests [65].

There is a need for greater consensus on socio-economic proxies (occupation, education, income, disadvantage measures) to facilitate meta-analyses and subgroup analyses. This is necessary for progress towards governments’ socio-economic equity goals [66,67,68] as reflected in the UN SDGs [to eradicate poverty (#1); achieve good health and well-being for all (#3); reduce inequalities (#10)] [69].

5. Conclusions

This systematic evaluation of population health evidence syntheses confirmed deficits in evidence through insufficient and inconsistent practice in reviews and primary studies, which ultimately limit evidence-based public health to address health inequity. Many reviews did not have an explicit equity focus and were often limited to the consideration of age, gender or sex and place. Few were able to complete a subgroup analysis to examine differential health outcomes/intervention effects, often prevented by a small number of primary studies or a lack of detail therein. This highlights missed opportunities for learning about health equity impacts, which could be improved through the routine use of PROGRESS-Plus for data extraction and equity consideration, with transparency regarding which indicators were/not reported and why.