Patient-reported experience measure (PREM) for patients with interstitial lung disease (ILD): modification of a pre-existing measure

TL;DR

This study adapts a patient experience measure originally designed for rheumatoid arthritis to be used with patients who have interstitial lung disease.

Contribution

The paper introduces a modified version of the RA-PREM tailored for ILD patients, validated for acceptability and face/content validity.

Findings

Four statements across three domains of the RA-PREM were modified after patient feedback.

The modified ILD-PREM was accepted by patients and met face/content validity criteria.

Seventy-three patients completed the ILD-PREM, showing its feasibility in clinical settings.

Abstract

Patient-reported experience measures (PREMs) are a key component of healthcare accountability frameworks, health policy, integrated care board commissioning and integrated care partnerships generating data which are crucial markers of patient care quality. The Rheumatoid Arthritis Patient-Reported Experience Measure (RA-PREM) incorporates the eight core elements of NHS Patient Experience Framework and is validated in a range of rheumatic conditions. Our objective is to determine the acceptability and feasibility of the RA-PREM for an interstitial lung disease (ILD) population.

A mixed-methods patient-centred approach incorporating an interdisciplinary research steering group with patient partners. Patient surveys evaluated the language and meaning of the RA-PREM 8 domains, 24 statements and response categories. A patient focus group examined contentious statements. A consensus group of expert patient-partners agreed statements for the modified RA-PREM. Focus group participants reviewed the modified instrument (ILD-PREM) for acceptability and face/content validity.

A single NHSE-commissioned, regional ILD service/UK.

Thirteen patients (10 male) diagnosed with ILD participated in focus group discussions. Critical discussion of the RA-PREM resulted in nuanced modifications of four statements of three domains. Five patients (three male) and three healthcare researchers attained consensus on the face/content validity of statements. Seventy-three patients completed the ILD-PREM following outpatient contact.

The ILD-PREM retains 24 statements representing the eight domains of the RA-PREM. It meets face/content validity criteria and is acceptable to an ILD population. Longitudinal validation of the ILD-PREM across ILD services including further testing in global minority groups will establish criterion and construct validity and objective measures of reliability.

Figures

Methodological approach. ILD, interstitial lung disease; PREM, patient-reported experience measure.

Tables

Table 1. Eight indicators of the NHS Patient Experience Framework

Respect for patient-centred values and preferences	Welcome the involvement of family and friends
Coordination and integration of care	Information, communication and education
Emotional support	Transition and continuity
Physical comfort	Access to care

Table 2. Four amended statements of the RA-PREM

Domain 1: Your needs and preferences
Original statement	Whenever I attended clinic, I felt that I was treated respectfully as an individual
Modified statement	During my appointments, I felt that I was treated respectfully as an individual
Domain 4: Daily living and physical comfort
Original statement	If I have had an exacerbation (when my symptoms got much worse), I have been able to get help quickly
Modified statement	If my symptoms get much worse (eg, I experience an exacerbation or flare), I have been able to get help quickly
Domain 4: Daily living and physical comfort
Original statement	I feel that my lung condition is being controlled enough to let me get on with my daily life and usual activities
Modified statement	I feel that I have the right support to adapt and manage my daily activities around my changing lung condition
Domain 5: Emotional support
Original statement	I feel able to discuss personal or intimate issues about relationships with my health team if I want to
Modified statement	I feel able to talk about personal or intimate issues with my health team, if I want to

Table 3. Characteristics of respondents

Characteristics	Cohort	Respondents
Total numbers (n)	128	73
Gender, n (%)
Male		36 (49.3)
Female		21 (28.8)
Non-binary		0 (0.0)
Prefer not to say		1 (1.4)
Not answered		15 (20.5)
Age (years), n (%)
39 or less		0 (0.0)
40–49		1 (1.4)
50–59		2 (2.7)
60–69		18 (24.7)
70–79		39 (53.4)
80–89		9 (12.3)
90 and over		0 (0.0)
Not answered		4 (5.5)
Ethnicity, n (%)
White (British/Irish/Other white background)		70 (95.9)
Mixed (White and Black Caribbean or African/White and Asian/Other mixed background)		0
Asian/British Asian (Indian/Pakistani/Bangladeshi/Other Asian)		0
Black or Black British (Caribbean/African/Other Black)		0
Chinese		0
Other ethnic group		0
Prefer not to say		0
Not answered		3 (4.1)
Diagnosis, n
Idiopathic pulmonary fibrosis	77	51
Hypersensitivity pneumonitis	27	9
Connective tissue disease-associated ILD	3	3
Unclassifiable ILD	4	10
Don’t know/diagnosis uncertain	N/A	1
Other (please specify)	37	1
Not answered	N/A	6
Years since diagnosis, n (%)
Less than 6 months	N/A	7 (9.6)
Between 6 months and 1 year		18 (24.6)
1–2 years		14 (19.2)
2–3 years		7 (9.6)
More than 3 years		21 (28.8)
Not answered		6 (8.2)
Forced vital capacity (FVC) % predicted, n
30–50%	16
51–70%	54
71–90%	48	N/A
91–130%	29
Not available	1

Full text

Introduction

Interstitial lung diseases (ILDs) are characterised by irreversible inflammation and scarring of the lung tissue that is often progressive. Associated with high morbidity and mortality rates, symptoms of ILD include debilitating breathlessness, cough and fatigue.1 2 ILD is the primary pulmonary manifestation in rheumatoid arthritis (RA) and connective tissue diseases (CTDs), such as scleroderma (systemic sclerosis), inflammatory myositis (polymyositis and dermatomyositis), Sjögren syndrome and undifferentiated CTD3 and symptomology is often shared. Optimal healthcare requires palliation, slowing disease progression and improving or maintaining quality of life (QoL). The efficacy of this approach can be measured by clinical biomarkers and the metrics of serial data of patient-reported measures (PRMs). PRMs guide decisions in palliative care such as outcomes, lived experience, motivation or treatment preference.

The National Institute for Health and Care Excellence Quality Standard, endorsed by NHS England, recommends follow-up appointments for patients with idiopathic pulmonary fibrosis (IPF)4 every 3–12 months, adjusted according to disease stability.4 5 This standard for IPF, the most common form of ILD, may be extrapolated to all ILDs. The clinical review minimally includes physical assessment, pulmonary function tests with the opportunity for oxygen assessment, appropriate onward referral for pulmonary rehabilitation, lung transplantation and/or palliative care4 and self-reports of symptoms, their impact and psychosocial status using Patient-Reported Outcome Measures (PROMs). PROM metrics are a component of healthcare quality frameworks offering population oversights that identify health inequalities with potential to improve health outcomes.6 Further, this includes how patients and their family and friends experience healthcare episodes, captured by patient-reported experience measures (PREMs) and the effective use of financial resources.

The metrics of patient experience underpinned by a philosophy of patient-centred care influences the decision-making of healthcare providers, integrated care boards and key stakeholders, further impacting service provision.7 We cannot improve aspects of health services if we do not collect and evaluate these metrics. The NHS Patient Experience Framework identified eight indicators to guide measurement of what is reported to be the most important elements of patients’ healthcare experiences (table 1).8 9

Properties of PREMs

PREMs use Likert scale scoring to quantify a range of care quality, including relational and functional elements of healthcare experience. The relational PREM statements focus on the patient–clinician relationship while functional PREM statements focus on the objective experiences of care facilities, for example, access, cleanliness or comfort. PREMs, like the other PRMs may be generic or disease specific10 and may have utility across a range of health and social care settings. The first stage in developing any new instrument is to evaluate pre-existing instruments and establish the need for a generic versus disease-specific instrument.

Three systematic reviews (SRs) examined the psychometric testing of PREMs. The first in a mixed inpatient population,10 the second in emergency care service provision11 and the third across a range of healthcare contexts.12 Beatie et al10 identified 26 papers for inclusion, including 11 international instruments. Male et al11 identified 8 papers measuring patient experience of emergency hospital care including 4 international instruments and Bull et al12 identified 88 PREMs in 109 papers meeting inclusion criteria across four main healthcare contexts of inpatient care services (36.4%), primary care (23.9%) and outpatient care services (12.5%). Fifty-nine of the PREMs were generic measures. Authors report evidence of extensive theoretical/development work and while the quality of methods and results was variable, mostly the standard was high. Bull et al12 evaluated the PREMs using the COSMIN framework reporting on validity and reliability, specifically internal consistency, structural and content validity and identified a need for greater rigour in the psychometric evaluation of each PREM.13

Given the number of PREM instruments in circulation, we were interested in those used or developed for use in conditions clinically associated with ILD. The Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS),14 29-item Generic PREM specifically for inpatient care, was reported to have utility in patients with chronic obstructive pulmonary disease (COPD) admitted with acute exacerbations.10 The HCAHPS was the preferred PREM in the CICERO–ELF patient survey of 200 participants.15 The HCAHPS is included in the Hospital Value-Based Purchasing programme to calculate value-based incentive payments in the USA and is focused on the perspective and experiences of hospital admissions.

The PREM-C9, a COPD-specific PREM captures patients experiences and interactions with healthcare systems and clinicians in three areas of healthcare: ‘usual care’ (items 1–3); ‘everyday life’ (items 4–7); and ‘self-management and exacerbations’ (items 8–9). The PREM-C9 has shown good reliability and known groups’ validity, suggesting it is appropriate for use as a measure of overall COPD experience using the overall score.16

The Rheumatoid Arthritis Patient-Reported Experience Measure (RA-PREM) developed by the CQRA (Commissioning for Quality in Rheumatoid Arthritis) used patient-centred methods.17 Focus groups identified key elements of the patient experience which researchers mapped to the NHS framework.8 The Cronbach’s alpha ranged from 0.76 to 0.91 across its eight domains demonstrating good internal consistency of the RA-PREM in 524 patients across 10 UK sites. The RA-PREM validated in the original study18 has subsequently been validated in other global rheumatology populations demonstrating feasibility, acceptability, relevance, validity and reliability.19 20

We considered the merits of the existing PREMs, both generic and disease-specific, and utility in IPF/ILD. The lack of appropriate PRMs, including PREMs, in IPF has contributed to gaps in the field.18 21 The Patient Experiences and Satisfaction with Medication Questionnaire (PESaM) has been used to quantify medication experiences (specifically Pirfenidone) in IPF,22 but currently, there are no PREMs available for clinical use.23

Two candidate instruments, the PREM-C9 and RA-PREM, were evaluated by our interdisciplinary research group for the adaptation for use in an ILD population. A preference for the RA-PREM was established and we were granted permission to adapt the RA-PREM to an ILD population.

Aims

To establish acceptability and feasibility of modification of the RA-PREM for an ILD population.To modify the content and language of the RA-PREM as indicated using patient-centred and consensus methods.To produce an acceptable PREM for validation in an ILD population (ILD-PREM).

Methods

See the infographic of the methodological approach in figure 1.

Methodological approach. ILD, interstitial lung disease; PREM, patient-reported experience measure.

Feasibility

Patient-centred researchers (JM, AD, A-MR) convened an ILD steering group of patient-partners, physicians, nurses and allied health professionals (n=10) to discuss the feasibility and utility of the RA-PREM for an ILD population and determine the need for modification.

Electronic survey

The research team adapted an electronic version of the RA-PREM, changing explicit references to RA disease to lung disease and distributed it to members of the ‘Exeter Patients in Collaboration for Pulmonary Fibrosis Research’ (EPIC-PF) group of patients diagnosed with ILD across a range of severity and trained in critical appraisal of research (n-14). EPIC members agreed to comment on the content, language and relevance of each item. Subsequently, the EPIC group convened a meeting to review comments.

Focus group

Thirteen patients (male=10) diagnosed with ILD participated in a 2-hour online focus group to discuss all statements of the RA-PREM. Each statement was read in turn, and comments were invited as to relevance, perceived meaning and need for modification. Discussion focused on incongruous statements and the proposal of amendments. Two female researchers (JM and A-MR) experienced in qualitative research methods and analysis, facilitated the audio-recorded group and kept field notes. We did not generate transcripts, rather listened and relistened to the recording to augment field notes and identify items that required modification.5

Refinement

Four expert patients (male=3) who had an established working relationship with the research team participated in cognitive interviews to achieve consensus of the meaning of modified items via an electronic platform. The penultimate version of the modified PREM (ILD-PREM) was circulated to the EPIC group and research team for further comment or final approval.

Pilot testing

Having agreed the final ILD-PREM, focus group members reviewed its acceptability and face/content validity. A pilot study evaluated the acceptability of the ILD-PREM at a single-centre, specialist ILD service. The ILD-PREM was sent to all newly referred patients attending the outpatient clinic over a 6-month period (August 2023 to February 2024) using a consecutive sampling approach. At the consultation, the researchers gave consenting patients a paper copy of the ILD-PREM with instructions on how to complete it and how to return the form anonymously in a prepaid envelope within 2 weeks of their appointment.

Patient and public involvement

Patient involvement was integral to this work and patient-partners were involved from initial conversations about feasibility, through to focus group participation, refinement and pilot testing. The EPIC group are an established group of patient partners in research.

Statistical analysis

Descriptive statistics were collated for patients’ demographics and clinical characteristics. Cronbach’s alpha calculated the internal consistency of the ILD-PREM for 20 qualifying questions. Questions 7b and 7d require binary response options and participants only complete questions 7c and 7e if they answer ‘yes’ to 7b and 7d. We excluded these four questions from the Cronbach’s alpha calculation as they did not meet all the assumptions of this statistical test. Descriptive statistics were collated using Microsoft Excel (V.2507), and Cronbach’s alpha was calculated using SPSS Statistics V.28.

Results

The interdisciplinary ILD steering group (n=10) unanimously agreed the appropriateness of the RA-PREM, its robustness and usefulness in modification.

The EPIC group members (n=10) reviewed the electronic RA-PREM commenting on each item. In turn, the steering group members examined each item and response. Where most comments were positive or neutral (Clear and easy to answer, pertinent and phrased well, highly relevant—well put No comment), the original statements were retained.

Four statements consistently generated critical comments. Criticism ranged from being purely practical, for example, one statement referenced ‘attending clinic’ and respondents identified that clinics are now often virtual. They considered other statements to be irrelevant or poorly worded in the context of ILD. For example, the statement ‘I feel that my lung condition is being controlled enough to let me get on with my daily life and usual activities’ generated the following responses: This is not relevant for a progressive condition, This is a bit vague! Not my usual activities before I had IPF, Not controlled enough could either be the nature of the condition or the support being given.

The steering group convened a focus group to examine these four statements. Focus group members proposed modification and alternative content or language for the four statements. One participant commented on the possibility of control of symptoms of diseases such as IPF no amount of treatment will help what I have.

There was recognition that ‘daily life’ or ‘usual activities’ vary, and the presence of a progressive and irreversible lung disease may permanently alter functioning. Adapting to what our patient group described as this new normal was a key component, alongside recognising that change was inevitable within this disease context.

Post focus group, researchers (JM; A-MR) compared field notes and redrafted the statements accordingly. We circulated the revised statements to the expert patient group, for refinement and clarification of meaning and consensus discussion (table 2).

The final version of the questionnaire was circulated to the expert patient group EPIC-PF, who agreed that the questionnaire was now appropriate for use in an ILD population, being clear, focused, well laid out and easy to understand.

We evaluated the ILD-PREM at a regional specialist ILD centre covering a diverse geographical catchment area in the Southwest of England. One hundred and forty-eight patients attended 28 clinics during the 6-month test period (137 face-to-face, 11 via telephone). Patients had waited for their initial assessment an average of 5 months, following referral (range: 1–10 months).

One hundred and forty-eight patients agreed to complete the questionnaires with 73 returned (response rate, 49.3%). Thirty-six respondents were male, 21 identified as female with 16 respondents not declared. While the most common ILD diagnosis, IPF was the most disproportionately represented diagnosis (n=51), and it was therefore unsurprising that with more than half of respondents were in the age range 70–79 years (see table 3).

Response option frequencies are available in online supplemental table S4: distribution of ILD PREM responses. We note the data are positively skewed, but do not make inferences, as there is no comparative measure.

Of the 73 questionnaires returned, 59 statements contained ‘no answer’ response. These missing data equate to 3.4% of the total number of questionnaires returned. Questions most frequently not answered were:

(7d) I have needed extra treatment or a change of treatment (between routine clinic appointments) (n=9 respondents). Inference: this may relate to the fact that this was patients’ first appointment.(4b) If my symptoms get much worse (eg, I experience an exacerbation or flare), I have been able to get help quickly (n=9 respondents). Inference: it is possible that the respondents may not have experienced an exacerbation.(7b) I have had appointments cancelled unexpectedly (n=7 respondents).(5a) I feel able to approach a member of my health team to discuss any worries about my condition and my treatment or their effect on my life (n=5 respondents)

These questions all appear on the second page of the questionnaire and we hypothesise a proportion of the respondents may have overlooked the page. Given the commitment to anonymity, we are unable to definitively establish the reasons for the unanswered questions. Evidence for structural validity of a scale is a prerequisite for interpretation of internal consistency analysis, a measure of internal structure of an instrument. Cronbach’s α ≥0.70 is considered indicative of sound reliability for subscale items as well as overall scores. The Cronbach’s α was 0.78 for total scores of completed ILD-PREM questionnaires (n=51), excluding the four questions listed above from this analysis and indicating a good internal consistency for the overall scale.

Discussion

Three SRs evaluating the psychometric properties of PREMs included 122 papers across multiple settings. Collectively, the SRs concluded that PREMs were acceptable to context, but the evidence on the ease of use was poorly reported.1012 While content validity and theoretical development were well reported, responsiveness was often absent.10 The authors’ evaluation concluded that the validity and reliability of PREMs should be reassessed throughout development.

The PREM-C9 was assessed for validity and reliability during initial development and underwent more extensive psychometric in the Catalan and Spanish versions.16 The PREM-C9 showed good reliability and known groups’ validity, appropriate for use as a measure of overall COPD experience using the overall score.16 Although translated in other languages, use of the PREM-C9 is only reported in COPD, its function being to measure the domains of everyday life, usual care and exacerbations across the spectrum of severity of disease. Our interdisciplinary research group critically evaluated the C9-PREM for potential adaptation for use in an ILD population. Their unanimous preference was for the RA-PREM as it offered standard closed-ended questions, ease of use and time efficiencies with data easy to extract for comparison across different settings. The comments section offered narrative information with ‘supplemental value’ on detail and context, and we determined the RA-PREM17 to be culturally and clinically relevant.

Using patient-centred approaches, we modified the RA-PREM17 to ensure the language and content were relevant to an ILD population. The 24 statements retained have linkage to the eight indicators within the NHS Patient Experience Framework.8 The final version of the ILD-PREM is found in online supplemental table S5.

Two statements, relating to domains 1 and 5, required only minor changes following patient feedback. Two statements, both linked to domain 4 (Daily Living and Physical Comfort) needed substantial rewording, not surprisingly in the context of ILD. The RA-PREM statement refers to a ‘flare’, which is a concept well understood in rheumatology. Although a contentious issue within ILD, being poorly defined and lacking clinician consensus and standardised patient understanding, we changed ‘flare’ to the more widely used term ‘exacerbation’. Patients with lived experience perceived this broader definition of decline to encapsulate ‘symptoms getting much worse’.

Likewise, within the next statement, ‘control’ is an appropriate treatment goal for rheumatological conditions, but not a realistic aim for the insidious, complex and frequently progressive symptoms associated with ILD, particularly breathlessness. The result was a much more nuanced statement, referring to support in adapting and managing daily activities. The modified statement recognises that change is inevitable, thereby removing the word ‘usual’, which is unhelpful. Full ‘control’ is unrealistic, and the passive statement (‘being controlled’) becomes an active one.

Our work demonstrates that modifying validated measures in this way, where there is reasonable homogeneity in disease course and symptom burden is a feasible and efficient way of developing measures for a different patient group. This approach is efficient and maximises the original research investment of time and resource. The work demonstrates the importance of recognising the complexities relating to chronic disease and the value of disease-specific measures16 17 23 as well as the importance of patient involvement in the development of measures.24 Patients offer a unique perspective on the relevance and language of questionnaire content, helping to ensure tools accurately measure what we think we are measuring, increasing the quality of our data (face and content validity).

Recent systematic reviews have highlighted complex unmet needs in this patient population, including a lack of psychological support throughout the course of living with pulmonary fibrosis.25 26 Misconceptions and barriers to the provision of palliative care and a gap between informational needs and provision prevail.26 The introduction of oxygen is a significant event both psychologically and domestically.25 Collecting PREM data at each encounter will offer insight at a population and service level informing targeted interventions to improve and reduce the burden of unmet needs.

Limitations

While this pilot testing of the ILD-PREM indicates acceptability of the measure within a specific population, further work will assess the acceptability of the ILD-PREM in more diverse groups. This includes not only patients from global majority ethnicities under-represented in our data but also patients with a wider range of ILD diagnoses. IPF is the most common form of ILD, accounting for 20%–50% of cases.2 Our demographic data show a disproportionate representation, at 70%. Further study will explore validity and additional reliability testing to establish the ILD-PREM appropriately captures patient‐reported experiences of healthcare across the range of ILDs.

Our preliminary work utilised the ILD-PREM within a single clinical setting, capturing data from newly referred patients on their initial, diagnostic visit. Further work is required to validate the measure longitudinally and across a range of clinical settings.

All PRMs, including PREMs, have limited usefulness in the absence of data. Missing data often result from non-responders, including those with low literacy levels, limited English or poor health. We distributed the ILD-PREM in paper format, but future work will likely require a hybrid approach with the option for an e-PREM. Exploring non-response patterns was beyond the scope of this preliminary work; however, this will be important in future work to address escalating health inequalities.

Conclusion

PREMs are useful tools to monitor and, subsequently, improve patient experience of care. Using patient-centred methods, we have modified the RA-PREM for use in an ILD population. The ILD-PREM retains 24 statements, which are representative of the eight indicators the PREM seeks to measure. Our expert patient group has confirmed face and content validity of the ILD-PREM. Participants in the pilot study confirmed acceptability of the ILD-PREM. Longitudinal studies will provide a larger data set to enable further psychometric testing using exploratory factor analysis and calculations of median time of completion to ensure that time constraints is not a barrier to completing the ILD-PREM. Objectively measuring patient experience within the wider ILD community will add to the quality of clinical care.

Supplementary material

10.1136/bmjresp-2025-003330online supplemental file 1

10.1136/bmjresp-2025-003330online supplemental file 2