Can we enhance neurorehabilitation through regional implementation of group-based telerehabilitation? A mixed-methods evaluation of NeuroRehabilitation OnLine (NROL)

TL;DR

This study evaluated a group-based telerehabilitation program, NROL, and found it successfully enhanced neurorehabilitation services in a regional healthcare system.

Contribution

The study demonstrates the successful regional implementation and sustainability of a group-based telerehabilitation model for neurological patients.

Findings

NROL was adopted by all intended organizations and continues as part of usual care with growing participation.

The program was acceptable to both therapy staff and patients and supported increased therapy provision.

Sustainability was achieved through staffing and travel efficiencies via collaborative regional systems working.

Abstract

To determine whether neurorehabilitation can be enhanced through regional implementation of group-based telerehabilitation, we implemented the NeuroRehabilitation OnLine (NROL) innovation regionally and evaluated scale-up from a systems perspective.

Observational, exploratory service evaluation using a mixed-methods convergent parallel design.

Stroke and neurological rehabilitation services from four organisations across a regional healthcare system in the UK.

Therapy staff from community-based services and patients with a stroke or neurological condition receiving active community rehabilitation including NROL from April 2022 to March 2024.

A regional multidisciplinary group-based neurological telerehabilitation innovation (NROL).

Selected Proctor’s implementation outcomes, to establish system-level adoption, acceptability and sustainability of the regional NROL innovation.

NROL was adopted by all intended organisations and continues as part of usual care with participation growing. It was acceptable to therapy staff and patients across the region, well-used, valued and supported increased therapy provision. For sustainability, staffing and travel efficiencies were identified through effective collaborative regional systems working. The importance of continued wide stakeholder engagement, robust evaluation and alignment was highlighted.

NROL was successfully embedded into real-world practice at a system level and enhanced neurorehabilitation. Looking forward, longer-term sustainment of this innovation will require a compelling business case and value proposition for decision-makers, addressing economic, equality and operational efficiency considerations.

Figures

NROL evaluation mixed-methods convergent parallel design. Service and survey data were collected and analysed separately. Results relating to the system-level indicators were synthesised under selected implementation outcomes and summarised using a joint display. CFIR, Consolidated Framework for Implementation Research.36 COM-B, Capability, Opportunity, Motivation-Behaviour Framework.40 NROL, NeuroRehabilitation OnLine.

Regional NROL organisation and patient participation profile. Profiles of participation across community-based stroke and neurological rehabilitation services of the four organisations within the regional healthcare system (East Lancashire Hospitals NHS Trust (ELHT), University Hospitals of Morecambe Bay NHS Foundation Trust (UHMB), Blackpool Teaching Hospitals NHS Foundation Trust (BTH) and Lancashire and South Cumbria NHS Foundation Trust (LSCFT)). *Intention to treat analysis used for patients who dropped-out after starting NROL (i.e. attended at least 1 targeted therapy group). NROL, NeuroRehabilitation OnLine.

Violin plots of therapy provision. Violin plots showing distributions of total therapy minutes received (A) and total days in service (B). Representations are for participants with stroke who received NROL as part of their rehabilitation (‘NROL’) and those who did not (‘Non-NROL’). Plots show the range, IQR, median and the density (or frequency) of data points across values, with wider sections indicating more participants with data at those values. NROL, NeuroRehabilitation OnLine.

Joint display merging mixed-methods data. NROL, NeuroRehabilitation OnLine.

Tables

Table 1. Overview of evaluation components

Data	Mode of collection	Source	Implementation outcome (Proctor’s)	System-level indicator	Indicator question	Operationalisation of indicator	Analysis approach
Service data (quantitative)
NROL service data	Routinely collected NROL operational data	NROL database (April 2022–March 2024)	Adoption	Participation	Who was involved in NROL delivery (organisations, staff)?	Number of organisations (n) Number of therapy staff (n), characterised by organisation and role	Descriptive statistics
					How many patients were referred to and participated in NROL?	Number of patients referred and participated across organisations (n), with reasons for withdrawal before starting (‘Did not start’) or drop-out (after at least one targeted therapy session) recorded Annual referral growth (%)	Descriptive statistics
				Delivery	What was delivered for NROL?	Number: blocks, groups, sessions (including cross-organisational delivery), patient and therapy staff contacts (n)	Descriptive statistics
			Acceptability	Utilisation	Was the NROL targeted therapy group allocation utilised by staff?	Median block capacity (%) (ie, median proportion patient allocation opportunities filled at start each NROL block)	Descriptive statistics
					Was NROL utilised by patients?	Rates of withdrawal before starting ('Did not start’) or drop-out (after at least one targeted therapy session) (%) Median number of sessions per patient (n) Patient session attendance rate (%) Number of blocks per patient (n)	Descriptive statistics
				Safety	Was NROL safe for patients?	Incident record	Descriptive reporting
			Sustainability	Efficiencies	Were staffing efficiencies provided?	Median staff:patient ratio	Descriptive statistics
					Were travel efficiencies provided?	Avoided: driving distance (miles), driving time (hours), mileage cost (£) and carbon emissions (kgCO2e)*	Descriptive statistics
				Sustained appropriateness	Were NROL participants representative of patients referred to NROL?	All patients referred to NROL characterised by demographic and clinical data (organisation, age, gender, ethnicity, deprivation, rurality, living alone status, condition, chronicity) Comparative analysis (between-subject) identified any significant difference or association in characteristics between patients referred to 'NROL’ who participated vs those referred to NROL but ‘Did not start’	Descriptive and inferential (Mann-Whitney U, χ2) statistics
					Were NROL participant outcomes (health-related quality of life, activity performance) consistent with those reported in the single-service evaluation?	Group-level: Comparative analysis (within-subject) identified any significant change between paired differences of entry and exit EQ-5D-5L (EQ-Index and EQ-VAS) and PSFS for NROL participants5459* Individual-level: Proportion (%) of NROL participants who improved or remained stable for EQ-Index, EQ-VAS and PSFS (change score (exit–entry) ≥0), and % who met or exceeded the MCID Minimally clinically important differences (MCIDs), defined as EQ-Index ≥0.10, EQ-VAS ≥10, and PSFS ≥2.8 points57 59	Descriptive and inferential statistics (Wilcoxon Signed Rank) statistics; comparison with MCID
Wider service data	Routinely collected stroke and neurological service data	Stroke and neurological service records (April 2023–March 2024)	Adoption	Participation	What proportion of wider service therapy staff was involved in NROL delivery?	Proportion of wider service therapy staff involved in NROL delivery (%)	Descriptive statistics
					What proportion of the total service patient caseload was referred to NROL?	Proportion total patient caseload referred to NROL (%)	Descriptive statistics
		Stroke clinical audit datasets (for Sentinel Stroke National Audit Programme,50 acquired from two organisations (ELHT, BTH) (April 2022 –March 2024)	Acceptability	Therapy provision(small-scale review)	Did NROL support increased therapy provision?	Comparative analysis (between-subject) identified any significant differences in total therapy amount received (minutes) and duration in service (days) between stroke patients who participated in NROL ('NROL’) vs those who did not (‘Non-NROL’) in total across four care domains (occupational therapy, physiotherapy, speech and language therapy and psychological support (psychology)) and within each of the four care domains*	Descriptive and inferential comparison (Mann-Whitney U) statistics
			Sustainability	Sustained appropriatenes (small-scale review)	Were NROL participants with stroke representative of patients with stroke within the wider services	Comparative analysis (between-subject) identified any significant difference or association in selected characteristics (age, gender, prestroke disability (mRS), admission stroke severity (NIHSS), deprivation and rurality) between stroke patients who participated in NROL ('NROL’) versus those who did not (‘Non-NROL’)*	Descriptive and inferential (Mann-Whitney U, χ2)
Survey data (quantitative and qualitative)
Service staff survey data	Cross-sectional service staff survey; online, MS forms	Service staff survey records (June–July 2023)	AdoptionAcceptabilitySustainability	Challenges and enablers, perceived value, considerations for sustained use	What were staff perceptions of NROL?	Quantitative: Agreement for rating questions (%) Qualitative: Themes, with illustrative quotes, derived from free-text responses coded deductively under CFIR headings with one main coder, one reviewer (SA, AP)	Descriptive statistics; CFIR-guided framework analysis
NROL patient survey data	Cross-sectional NROL patient survey; telephone	NROL patient survey records (April 2022 - August 2023)	AdoptionAcceptabilitySustainability	Challenges and enablers, perceived value, considerations for sustained use	What were patient perceptions of NROL?	Quantitative: Agreement for rating questions (%) Qualitative: Themes, with illustrative quotes, derived from free-text responses coded deductively under CFIR headings with one main coder, one reviewer (SA, AP)	Descriptive statistics; CFIR-guided framework analysis

Table 2. NROL delivery, utilisation and efficiency summary

	April 2022–March 2024
Service metric	Regional total
Total: Delivery and staffing efficiency
Blocks delivered, n	12
Sessions delivered, n	511
Patient contacts, n	3434
Therapy staff contacts, n	1167
Therapy staff:patient ratio, median (range)	0.4 (0.1–2.0)
Targeted therapy groups: Utilisation
Block capacity, median %	84%
Patient session attendance rate, %	71%

Table 3. System-level key constructs influencing NROL implementation

Construct		Summary	Quantitative survey data agreement (%) (where relevant and available)	Qualitative survey data illustrative quotes
	Individuals (staff and patients)
Motivation/capability	Staff	Across the region, therapy staff felt positive about NROL and felt supporting delivery was acceptable, with most feeling confident to do so. Aspiration for sustainment.	82% felt positive about NROL and felt it was clear how NROL helps to improve rehabilitation delivery. 84% felt supporting the delivery of NROL in their service was acceptable, with 78% feeling confident to do so.	UHMB: “Brilliant addition to the range of services available, time saving, cost saving and great to be able to work with others in, and network in, the region.” BTH: “Huge—benefits clients’ rehab journey, offers peer-support.” UHMB: “It is a really great service and I think the more it is adopted it will just become the norm and a great adjunct to other services that are out there.”
	Patients	Across the region, patients felt positive about NROL as a valuable and convenient part of their rehabilitation. Some saw potential for more use.	93% thought NROL was a valuable part of their rehabilitation. 94% felt it was convenient from a time and energy point of view. 85% felt that the peer support they received was important.	LSCFT: “Very glad I did it, it’s gave more confidence going forward.” ELHT: “Invite more people. Roll it out across the country. Spread the word.”
Opportunity	Staff	NROL was valuable for increasing therapy provision and practice (amount and wider range of therapy, out-of-session practice).	73% felt offering NROL increased the intensity of rehabilitation for their patients.	LSCFT: “Increases access to wider range of therapies.” UHMB: “It’s great at increasing therapy sessions to our patients and provides motivation outside of face to face sessions.” ELHT: “Allowing patients to access additional therapy which might be difficult to deliver due to limited service provision or difficulty to deliver the intensity needed.”
	Patients	Most patients that participated felt they had the necessary resources and support.	97% had all the equipment and space they needed to take part in NROL. 98% felt they received all the support they needed to access NROL online and 96% felt the staff that ran NROL sessions paid attention to their needs.	UHMB: “No special equipment was required.” BTH: “I felt seen, heard, understood and safe …The work done by NROL is formidable, it has given me confidence and courage to keep trying.”
	Innovation (NROL)
Complexity	Staff	The effort of implementation was deemed worth the benefit.	64% felt NROL required a lot of effort to implement in their service, however 71% felt the effort to implement NROL was worth the benefits.	ELHT: “Initial challenges balancing team priorities/staff capacity but now more established… benefits have outweighed this.” UHMB: “I do like NROL and its ethos … It can feel time consuming to deliver but if that's related to positive patient outcomes then it's worth it!”
Relative advantage	Staff	Improved patient outcomes through structured targeted therapy.	73% felt NROL improves outcomes for patients.	ELHT: “Many of my patients have improved on their outcome measures with NROL and it has helped me to provide a higher intensity of rehab particularly when staffing is short within our team.” ELHT: “Helps patients to regain weekly routines and structure to their day…” ELHT: “Improved outcomes both physically, cognitively and mentally.”
		Operational, staffing and travel efficiencies.	40% reported NROL reduced the travel requirements for their team.	LSCFT: “It is a good and efficient way of providing education and reinforcement of the rehabilitation/compensatory techniques provided by our team.” ELHT: “Efficient way of seeing patients, cost effective, seeing patient within their own environment.” ELHT: “Financial benefit with the reduction of travel times and costs of paying for travel.”
		Therapy staff increased technological skill and capability through NROL. Perceived overall professional development benefits.	29% reported their proficiency with digital technology increased after observation or involvement with NROL. 72% felt NROL benefitted their professional development.	UHMB: “Maintain a dialogue with other therapists/tech support to learn from each other.”
	Inner setting (healthcare setting)
Structural characteristics	Staff	Dedicated NROL operational team and champions deemed important.		BTH: “The IT support and NROL team in the background really help to make it work and help reduce staff time.” LSCFT: “It helps to have a champion to help proactively identify people who may benefit.”
		Challenge due to lack of shared regional technology infrastructure.		ELHT: “[Challenge] Documenting information, as not a central system.”
Relational connections	Staff	Collective use of workforce across region fostered collaborative learning and networking.	70% felt NROL helped their networking with other staff.	ELHT: “The chance to create therapy sessions with colleagues rather than our usual F2F alone.” LSCFT: “It is a good way for clinicians to liaise with and learn from other rehab teams/therapists.”
	Staff	Early and wide stakeholder engagement and gaining visibility was important.		ELHT: “Promote NROL to a wider range of therapist within a Trust to ensure the understanding and adoption of the service.” UHMB: “Engage all staff members early on to ensure that staff understand the concept.”
Compatibility	Staff	Challenge for therapy staff in balancing with their competing priorities.	42% felt supporting the delivery of NROL easily fitted in with other priorities.	LSCFT: “Balancing time between face to face clinical work and keeping space for NROL.” UHMB: “Challenge to fit into timetable due to annual leave, rota’d days off, other meetings.”
		Challenge with initial reluctance, and balancing workload fairly across organisations.		UHMB: “I have found a real struggle with getting other staff members on board … They do not seem to grasp that it will actually free more of their time after the initial time learning the new system.” ELHT: “Trusts have at times been difficult to get as readily involved in the delivery of the sessions, often reporting reduced confidence and requests for further observation.” BTH: “Every trust has to invest in all parts to make it work fairly.”
Access to knowledge	Staff	Opportunity for observational learning important		UHMB: “Observe sessions to build confidence before delivering.” LSCFT: “It helps to make shadowing a routine part of staff development so that people understand what actually happens in the groups.”

Table 4. NROL patient characteristics

	April 2022–March 2024
Patient characteristics	Regional total(n=339)
Age years, median (range)	60 (19–89)
Gender, male (%)	190 (56%)
Ethnicity
White	301 (89%)
Asian British or Asian	21 (6%)
Black British or Black	4 (1%)
Other	5 (2%)
Not reported	8 (2%)
Index multiple deprivation, median (range)	5 (1–10)
Rurality, rural (%)	79 (23%)
Living alone, yes (%)	57 (17%)
Condition
Sudden	310 (91%)
Stroke	257 (76%)
Progressive/intermittent	29 (9%)
Chronicity
Subacute (1 week to <6 months)	225 (66%)

Full text

Introduction

Multidisciplinary rehabilitation plays an important role in minimising disability, maximising function and optimising quality of life for people living with long-term neurological conditions. Higher doses of neurorehabilitation improve outcomes.14 In England, despite agreed recommendations for therapy amount,5 6 actual provision falls significantly below recommended levels, with variable access across providers7; a pattern seen internationally.8 9 Barriers to guideline adherence include workforce challenges, inefficiencies in resource allocation and lack of supporting organisational processes.1012 Achievement of recommended targets for access and provision may remain unmet until transformative system-level change is undertaken;11 12 however, solutions need to be feasible within our existing healthcare settings.

Telerehabilitation, the delivery of rehabilitation remotely via telecommunication devices, is one solution that may help address shortfalls in therapy provision and is an approach endorsed for use alongside conventional in-person therapy.5 When delivered remotely, rehabilitation interventions have been shown to be safe and equally effective as those delivered in person, including for stroke and neurological populations for patient outcomes such as activities of daily living and health-related quality of life.1316 A scoping review of telerehabilitation implementation studies indicated that limited technical skills hinder implementation while predominant facilitators include patient motivation and leadership involvement.17 Patient and clinician uptake and satisfaction are facilitated by appropriate training (technology use, supporting clinicians’ adaptation of practice for remote delivery) and the use of telerehabilitation to augment rather than replace in-person therapy.18 Telerehabilitation can minimise barriers to access, such as reducing travel requirements and optimising resource use, and can support standardisation of care while facilitating rehabilitation engagement.1921 These advantages may be especially important in areas with widely dispersed populations, a lack of local services and/or staffing shortages. As national healthcare services face reform, strategic drivers aim to shift resources to the community, improve productivity and flow, and integrate digital transformation initiatives.22 23 Incorporating acceptable telerehabilitation models into existing care pathways and systems may provide opportunity for enhancing rehabilitation with efficient resource use.24

NeuroRehabilitation OnLine (NROL) is a multidisciplinary, group-based, real-time telerehabilitation innovation that was successfully adapted from a pilot version25 and implemented at a single National Health Service (NHS) community stroke and neurological service in the UK.19 It leverages group-based therapy, which is supported by favourable evidence, benefits peer support and offers staffing efficiencies.19 25 Embedded within the service’s established neurorehabilitation care pathway, NROL contributed to the rehabilitation offer, sitting alongside existing therapy provision as part of a hybrid model of care. Supported by a small dedicated operational team, including technology support, it offered advantages to save time, energy and travel, enabling remote therapy to be delivered using existing workforce.19 NROL at this single-service level was deemed appropriate for patients with various neurological conditions and acceptable to patients and therapy staff, with positive outcomes and opinions. The innovation aligned with strategic priorities, including the use of data and digital technologies in healthcare.26 NROL was acknowledged in UK and Ireland stroke guidelines as an exemplar innovation for remotely delivered rehabilitation.5 To build on these positive results, a decision was made to expand NROL into a regional innovation.27 28

Scale-up of NROL was a ‘deliberate effort to increase the impact of this successful innovation to a greater number of services across the region’,29 and aligned with statutory changes for healthcare organisations to work cross-regionally to improve population healthcare, tackle unequal outcomes and access, and enhance productivity and value for money.30 Scaling up efforts should be underpinned by key principles including systems thinking and a focus on sustainability.29 31

Within a clinical-academic partnership, we sought to evaluate whether our regional community neurorehabilitation services could be enhanced through system-level implementation of the group-based telerehabilitation programme, NROL. Our objective was to evaluate the scale-up of NROL, embedded within established stroke and neurological rehabilitation pathways across multiple services at four healthcare organisations, from a system-level perspective. Implementation success was conceptualised by exploring the following questions:

As a regional innovation:

Was NROL adopted across the region?Was NROL accepted and a way to support increased therapy provision?What key considerations help inform NROL’s sustainability?

Methods

Real-world implementation of NROL was examined through an observational, exploratory, service evaluation. A mixed-methods design was used and is described below, preceded by an overview of context. This approach is particularly valuable when evaluating the implementation of complex innovations, where both measurable outcomes and contextual factors matter.32 Manuscript preparation was guided by the Good Reporting of a Mixed Methods Study.33

Implementation was explored using selected Proctor’s implementation outcomes of adoption, acceptability and sustainability,34 agreed with decision-makers (NHS managers and commissioners) as relevant, and within resources available. System-level indicators for the implementation outcomes were:

Adoption: NROL participation and delivery across organisations, and challenges and enablers to adoption.Acceptability: NROL utilisation, safety and perceived value for therapy staff and patients, and a way to support increased therapy provision.Sustainability: Key considerations for NROL to continue as usual care as a regional innovation, including model efficiencies (staffing, travel avoidance) and sustained appropriateness (representativeness of NROL participants to the patient population, NROL patient outcomes consistent with single-service NROL evaluation).

Implementation context

The regional NROL innovation was coproduced within a collaboration between a regional healthcare system (Lancashire and South Cumbria Integrated Care System) and a clinical-academic partnership (Lancaster University and East Lancashire Hospitals NHS Trust (ELHT)). ELHT, the host healthcare organisation, registered the NROL project as a service evaluation, with data routinely collected and anonymised. Implementation was guided by the updated Medical Research Council framework for developing complex interventions.28 32 From April 2022, a horizontal phased approach29 scaled NROL from a single-service (ELHT) to a multiservice model, integrating three further organisations (University Hospitals of Morecambe Bay NHS Foundation Trust (UHMB), Blackpool Teaching Hospitals NHS Foundation Trust (BTH) and Lancashire and South Cumbria NHS Foundation Trust (LSCFT)).28 Context was described using the Consolidated Framework for Implementation Research (CFIR), a commonly used determinant framework providing a menu of constructs under domain headings that have been associated with effective implementation.35 36 A summary is provided below under relevant domain headings, with full details published separately.27 28

Inner setting (healthcare setting)

Community-based stroke and neurological rehabilitation services from four organisations (ELHT, UHMB, BTH, LSCFT) within a regional healthcare system in the UK were involved. These services provide multidisciplinary rehabilitation for adults with sudden onset or progressive/intermittent neurological conditions. Before NROL’s scale-up (April 2022), services existing rehabilitation offer primarily consisted of in-person individual evidence-based therapy at patients’ homes, with limited onsite group and occasional online individual therapy sessions. The exception was ELHT, who had provided online group therapy via NROL at a single-service level since January 2021.

Innovation (NROL)

NROL as a regional innovation has been comprehensively described.27 Embedded into established neurorehabilitation care pathways, the NROL programme aimed to improve access to therapy by providing telerehabilitation to complement the existing rehabilitation offer. In brief, NROL delivered real-time (synchronous) group-based multidisciplinary evidence-based therapy online (Microsoft Teams) over repeating 6-week ‘NROL blocks.’ Each NROL block began with an entry session, participants then followed individualised programmes tailored to their needs, including weekly targeted talking (eg, cognitive, communication, well-being) and/or physical (eg, mobility, upper limb) therapy groups and an optional community peer-support group, for 6 weeks. Groups were delivered by therapy staff (allied health professionals, psychologists, assistants), patient volunteers and students. To facilitate participation, patients received set-up and ongoing assistance from a dedicated NROL technology support team member, who was present during sessions. Clinical reasoning determined if and when an NROL programme fitted within a patient’s overall rehabilitation. Patients typically exited the programme at the end of their 6-week block; however, they could attend more than one NROL block if clinically indicated, providing they remained under the active care of their stroke or neurorehabilitation team.

Individuals (patients and staff)

Patients undertaking community neurorehabilitation could be referred to NROL by therapy staff. In addition to service eligibility, inclusion required English proficiency (or translator access) and willingness to participate in online group therapy, with or without carer support. Participants required access to suitable computer hardware or connectivity, though support was sought from Age UK. Patients remained part of a local treating therapist’s caseload who was ultimately responsible for their overall rehabilitation. Existing therapy staff from all involved services delivered NROL collectively (ie, shared care approach), coordinated by a dedicated regional NROL team (operational manager, technology support, administrator). Clinical academics and organisational (NHS) neurorehabilitation managers provided leadership.

Outer setting (external influences)

NROL aligned with UK national healthcare strategic drivers including an ambition for equitable regional service delivery, managing workforce challenges, efficient resource use, digital innovation and delivering towards environmental targets,2223 26 3739 and is endorsed in UK and Ireland stroke guidelines.5

Design

A mixed-methods convergent parallel design was used to evaluate NROL scale-up and is overviewed in figure 1. Service data (quantitative) were collected throughout the evaluation period (April 2022–March 2024), with NROL service data analysed concurrently. Survey data (quantitative and qualitative) were collected from service staff (June–July 2023) and NROL patients (April 2022–August 2023) and analysed separately. Results relating to the system-level indicators were synthesised and interpreted under selected implementation outcomes and summarised using a joint display.

NROL evaluation mixed-methods convergent parallel design. Service and survey data were collected and analysed separately. Results relating to the system-level indicators were synthesised under selected implementation outcomes and summarised using a joint display. CFIR, Consolidated Framework for Implementation Research.36 COM-B, Capability, Opportunity, Motivation-Behaviour Framework.40 NROL, NeuroRehabilitation OnLine.

Data collection and analysis

Table 1 provides an overview of the evaluation components to demonstrate data alignment with relevant implementation outcomes and system-level indicators, and to outline the analysis approach. Additional details are provided below and in online supplemental file 1.

Service data were obtained: (1) NROL service data from the NROL database, inputted by the NROL operational team and group facilitators and (2) relevant wider service data from service records and stroke clinical audit datasets, provided retrospectively by service managers where possible. For NROL data, intention-to-treat analysis was applied to participants who dropped out (ie, after at least one targeted therapy session). Service data were predominantly summarised using descriptive statistics and reporting. Comparative analysis with inferential statistical testing was used to quantify therapy provision between patients with stroke who participated in NROL (‘NROL’) and those who did not (‘Non-NROL’, ie, wider service patients not referred to NROL or referred but did not start NROL). It was also used to evaluate sustained appropriateness (representativeness of NROL participants to the patient population, NROL patient outcomes (within-subject) consistent with single-service NROL evaluation).19 See table 1 for further detail.

Survey data were obtained from cross-sectional surveys undertaken by service staff and NROL patients within participating services across the region as part of service evaluation. Survey development was informed by the COM-B (Capability, Opportunity, Motivation-Behaviour) framework,40 and related to project outcomes. The COM-B was chosen as it recognises key factors integral in changing behaviour, is widely used in public health and aligns with the updated CFIR used for analysis. The surveys used a mix of multiple choice, Likert-scale rating and open-ended questions. The open-ended questions were used to provide deeper insights into implementation challenges and enablers, perceptions of NROL’s value, and key considerations for its sustained use. They were each piloted on 2–3 people to check content, clarity, usability and time taken and minimally refined to address feedback (see online supplemental file 2 for final versions). The staff survey link was shared to all relevant service staff via service management to collect anonymous feedback. Patients were surveyed via telephone as part of their NROL exit procedure and were aware that the survey administrator was not a member of the clinical team, and an honest perspective was wanted. Survey data were analysed using descriptive statistics for quantitative rating responses and CFIR-guided framework analysis for qualitative freetext responses. For each stakeholder group, quantitative and qualitative survey findings were merged and summarised under relevant CFIR headings to detail system-level constructs influencing implementation, with equal emphasis placed on both strands.

Analyses were performed using Microsoft Excel (V.2411), StataNow/MP V.18.5, and IBM SPSS Statistics V.29.0. All statistical tests were two-tailed, and statistical significance was determined at a p value threshold of 0.05. Where normality testing (Shapiro-Wilk test) indicated significant non-normality (p<0.05), non-parametric statistical testing was used and medians, with ranges or IQRs, reported.

Mixed-methods synthesis

A joint display41 was used to provide structure to summarise the service and survey analyses under the selected implementation outcomes, a recommended approach for presenting mixed-methods results42 and consistent with our prior reporting.19 Aligning service results with survey findings enabled synergistic insights when exploring system-level adoption, acceptability and sustainability. Insights were summarised by SA and LC and discussed and agreed by the clinical authors (AP, RP and HV) and the wider stakeholders involved in NROL.

Evaluator characteristics and reflexivity

This project was part of a clinical academic partnership, with LC and SA being both experienced researchers and physiotherapists in neurorehabilitation who have been involved in NROL development, implementation, evaluation and sustainment. AP is a physiotherapist who works as the operational manager of NROL for Lancashire and South Cumbria. RP is an occupational therapist at UHMB who delivers NROL. HV is a speech and language therapist and Head of Service for Stroke and Neurorehabilitation at ELHT and provides strategic leadership for NROL. TM is an applied econometrician working as a quantitative researcher focused on health and healthcare. The evaluators’ embedded role in NROL implementation provided valuable contextual insight. However, they remained mindful of potential bias and took steps to mitigate this. Evaluation was guided by logic models and informed by regular engagement with diverse stakeholders. Findings and interpretations were discussed and agreed with the wider stakeholder groups involved in NROL.

Patient and public involvement

Stakeholder engagement is a core element in developing and evaluating complex interventions.32 Patients and carers codesigned NROL within a learning collaborative, providing vital contributions to enhance its quality and relevance as previously described with a GRIPP2 reporting checklist.27 Involvement has continued throughout implementation, with patients providing feedback to refine content and delivery. Engagement has resulted in tangible improvements, for example, a trial of a ‘Vocation’ group and the creation of patient video vignettes to communicate key NROL information to diverse audiences in an accessible and meaningful way.

Results

Service data are reported over the total evaluation period (April 2022–March 2024). Online supplemental file 1 contains tables presenting data annually and from the preceding year of single-service NROL (April 2021–March 2022, ELHT), which enabled review of NROL delivery, utilisation and efficiencies over time.

NROL surveys were completed by 49 therapy staff (estimated response rate 36% of regional community neurorehabilitation therapy workforce) and 176 NROL patient (response rate 85% of NROL participants completing NROL in the survey time window). Respondents’ characteristics (online supplemental file 2) reflect regional therapy workforce and NROL patient characteristics and participation profile (online supplemental tables S1 and S7, figure 2).

*Regional NROL organisation and patient participation profile. Profiles of participation across community-based stroke and neurological rehabilitation services of the four organisations within the regional healthcare system (East Lancashire Hospitals NHS Trust (ELHT), University Hospitals of Morecambe Bay NHS Foundation Trust (UHMB), Blackpool Teaching Hospitals NHS Foundation Trust (BTH) and Lancashire and South Cumbria NHS Foundation Trust (LSCFT)). Intention to treat analysis used for patients who dropped-out after starting NROL (i.e. attended at least 1 targeted therapy group). NROL, NeuroRehabilitation OnLine.

Results are presented according to the system-level indicators for each implementation outcome.

System-level adoption: participation, delivery, challenges and enablers

NROL participation

Phased NROL adoption by all four organisations was completed across a 6 month period between April and September 2022 (figure 2). During the evaluation period, 75 existing multidisciplinary therapy staff from across the four organisations delivered NROL groups within their existing role (online supplemental table S1 for role and organisation detail), supported by three NROL operational team members. Per NROL block, about 40 therapy staff (~30% regional community neurorehabilitation therapy workforce) delivered sessions. Additionally, 167 staff registered to observe NROL. The NROL patient participation profile is depicted in figure 2: 438 patients were referred, reflecting approximately 2%–16% of each organisation’s wider service patient caseload (online supplemental table S2), with annual referral growth (Single-Regional Y1: 59%; Regional Y1–Y2: 30%, (online supplemental table S3); 339 patients participated (77%), with 293 (67%) of referred patients completing their programme.

NROL delivery

Key delivery data are summarised in table 2, with further data (including group-level data) provided in online supplemental tables S3 and S4. 12 NROL blocks were run successfully involving ten different groups (Cognitive education, Cognitive processing, Adjustment/well-being, Fatigue, Dysarthria, Aphasia, Vocation, Balance and Mobility, Upper limb, Peer-support). 69% of the sessions were run cross-organisationally (ie, therapy staff from ≥2 organisations). The proportion of therapy staff contacts from each organisation ranged from 7% to 53% (variation likely reflecting phased organisational participation and different service sizes); however, when considered relative to referral proportion, differences were between −6% and 7%. To support delivery, there were 488 NROL technology support contacts, 23 students and 19 patient-volunteer (for peer-support group) contacts. NROL remains part of usual care.

Challenges and enablers to adoption

Survey findings are summarised in table 3, detailing system-level key constructs identified as influencing NROL implementation. Content includes challenges and enablers to adoption, along with perceived value and future considerations as presented in subsequent result subsections.

Challenges to adoption included limited shared technological infrastructure, initial reluctance and difficulty balancing workload alongside competing priorities and across organisations. Staff also recognised that implementation took up-front effort, but that demands lessened as familiarity increased. Key enablers identified included staff confidence aided by observational opportunity to access knowledge about NROL, and structural characteristics such as leadership from a dedicated operational team (including technology support) and innovation champions. These ‘Inner setting’ constructs were reported by staff only, as patients are typically less aware of healthcare system factors.

System-level acceptability: utilisation, safety, perception of NROL and supported increased therapy provision

NROL utilisation

Key utilisation data for the targeted therapy groups are summarised in table 2, with further detail (including group-level data) provided in online supplemental tables S3 and S4. Blocks were run at a median of 84% capacity. NROL withdrawal (did not start) and drop-out (after at least one targeted therapy session) rates were 23% and 14% respectively (figure 2), and across organisations ranged from 20% to 28% and 2% to 18%. Participants attended a median of 9 sessions (range=1–50). Overall, the targeted therapy patient session attendance rate was 71% (group range 60%–83%), with organisation patient session attendance rate ranging between 65% and 83%. The peer-support group was optional to attend, and 200 patients (59%) chose to participate. Most patients participated in one NROL block (79%), but some participated in two (18%), three (2%) or four (1%).

Safety

One non-injury event was recorded: a minor seizure relating to a known condition during a talking session. NROL staff followed standard operating procedure, no first aid was required and the patient was able to continue participation.

Perceptions

Survey findings (table 3) demonstrated concordant positive feedback. The majority of therapy staff felt positive about NROL and found supporting delivery acceptable. NROL was perceived by many as a valuable opportunity for increasing therapy provision, practice and outcomes, with some identifying efficiencies. Implementation benefits were considered worthwhile to justify the effort to adopt. Additional value was perceived through professional development (including technology skills), as well as collaborative learning and networking. Patients across the region indicated high satisfaction with NROL. It was seen as a valuable and convenient part of their rehabilitation and they felt they had the necessary resources and support providing opportunity to take part. Both staff and patients reported motivational benefits, including the value of peer support.

Therapy provision

Figure 3 illustrates the distribution of total therapy provision for patients with stroke that were ‘NROL’ participants and ‘non-NROL’ patients. Small-scale review (n: NROL=76, Non-NROL=1520) demonstrated patients with stroke who participated in NROL received significantly more therapy (median therapy minutes received (IQR): NROL=1535 (1015–2855) vs non-NROL=335 (165–833), p<0.001). This reflected a significant increase in receipt of therapy for OT and PT domains (both p<0.001). Additionally, patients with stroke who participated in NROL had a longer duration in service (median days (IQR): NROL=146 (117–168) vs non-NROL=59 (28–109), p<0.001); with increased duration for the allied health professions (OT, PT, SLT all p<0.05)). Full details across the four care domains are provided in online supplemental table S5.

Violin plots of therapy provision. Violin plots showing distributions of total therapy minutes received (A) and total days in service (B). Representations are for participants with stroke who received NROL as part of their rehabilitation (‘NROL’) and those who did not (‘Non-NROL’). Plots show the range, IQR, median and the density (or frequency) of data points across values, with wider sections indicating more participants with data at those values. NROL, NeuroRehabilitation OnLine.

System-level sustainability: efficiencies, sustained appropriateness and future considerations

Efficiencies

Staffing efficiencies were achieved, with a median staff:patient ratio of 0.4 across all sessions (table 2), improving annually from 0.4 to 0.3 (online supplemental table S3). Group-level data demonstrated that all groups were delivered with a median ratio less than or equal to 0.8 (online supplemental table S4).

For travel avoidance, avoided driving miles totalled 75 554 (annual average=37 777 miles) and avoided driving time 2506 hours (annual average=1253 hours). Avoided mileage cost totalled £44 577 (annual average=£22 289). Avoided carbon emissions totalled 23 046 kgCO_2_e (annual average=11 523 kgCO_2_e). For annual details, see online supplemental table S6.

Sustained appropriateness

Representativeness of NROL participants to the patient population

Participating patient characteristics are presented in table 4, demonstrating varied patients accessed NROL. Patients’ residences were across an area circa 4480 km^2^, with the percent of regional NROL participants living rurally (23%) aligning with that of the region (20%).43 Compared with patients that were referred to NROL but did not start, patients that participated in NROL were younger (median age, year (IQR): NROL=60 (50–71) than those who were referred but did not start=63 (55–74), p<0.05) and more likely to have conditions that were sudden onset (p<0.01) and at the subacute phase (p<0.05). Small-scale review of selected characteristics indicated patients with stroke who participated in NROL were younger than those who did not participate (median age (IQR): NROL=61 (54–71) vs non-NROL=75 (64–82), p<0.001). They also statistically had lower level prestroke disability (mean mRS: NROL=0.41 (1.01) vs non-NROL=0.87 (1.19), p<0.001); however, both were less than 1 (where 0 indicates no symptoms and 1 indicates no significant disability and normal functioning) and the difference may be consistent with age differential. See online supplemental tables S7 and S8 for all comparisons.

NROL patient outcomes

Complete data sets were available for EQ-5D-5L and PSFS for 280 and 282 patients (83%), respectively. At a group level, statistically significant improvements were observed over time for both health-related quality of life measures (EQ-Index, EQ-VAS) and activity performance (PSFS) (all p<0.01). Large overlaps in IQRs and SD were noted, and no difference met the minimally clinically important difference (MCID). Overall, these results indicate a systematic positive shift in paired differences, rather than a substantial change in central tendency. At an individual level, the proportions of NROL participants who improved or remained stable were between 64% and 80%, with 30%–39% of participants meeting or exceeding the MCID. For full details, see online supplemental table S9.

Future considerations

Survey findings (table 3) highlight staff and patient aspiration for NROL sustainment and further use. Staff indicated the importance of relational connections, with early and wide stakeholder engagement and gaining visibility deemed important.

Mixed-methods synthesis

A joint display (figure 4) shows the convergence across service and survey analyses to explore the implementation outcomes of system-level adoption, acceptability and sustainability. Insights from the wider NROL stakeholders gathered through collaborative discussions shaping this paper highlighted the value of capturing and sharing robust evaluation and feedback, and underscored the necessity for strategic alignment across multiple levels. These perspectives have been incorporated into sustainability content.

Joint display merging mixed-methods data. NROL, NeuroRehabilitation OnLine.

Discussion

This observational mixed-methods evaluation of the system-level scale-up of NROL, a group-based telerehabilitation innovation, demonstrated enhancement of our regional community neurorehabilitation services. Despite anticipated healthcare system pressures, NROL was widely adopted and successfully embedded within real-world practice at all intended organisations across the region, with growing participation supported by collaborative system-level working. Acceptability was evidenced by the programme being well-used and valued by patients and therapy staff, while safely supporting increased therapy provision. Successful adoption, acceptability and model efficiencies indicate strong potential for sustained delivery, although continued stakeholder engagement and alignment remain essential for long-term sustainability.

A data-driven approach enabled real-time tailoring to optimise innovation-context fit, an important factor in scaling efforts.29 44 NROL proved compatible at a system level, with phased adoption across three additional organisations over 6 months. This led to growth, with increased referrals in the first year (59%) followed by annual growth (30%), while demonstrating improved staffing efficiency over time (staff:patient ratio 0.4 to 0.3). Despite this growth, referrals accounted for a relatively small proportion of the wider caseload (2%–16%), indicating potential for further participation. However, with block capacity nearing maximum (84%), broader participation may require model adjustments.

Implementation was influenced by individual and healthcare setting factors, such as initial therapy staff reluctance, competing priorities and infrastructure challenges, consistent with reported barriers to scale-up, innovation implementation and sustained use in healthcare.45 46 Nevertheless, key enablers facilitated success, in particular leadership from a dedicated operational team and innovation champions. The importance of highly committed individuals is well-recognised.4549 The impetus to overcome barriers may have been influenced by staffs’ recognition of the potential long-term benefits for patients, including improved outcomes for many and peer support. Consistent with staff perceptions and previous single-service NROL evaluation,19 quantitative results indicated positive shifts in participants’ activity performance and health-related quality of life, supporting the programme’s sustained appropriateness when delivered at a regional level. In line with the prior findings, clinically meaningful group-level differences were not detected, although meaningful gains were reported for some individuals (30%–39%). This may not be surprising given the relatively short duration of the NROL programme (typically 6 weeks) and mixed conditions. From October 2024, the EQ-5D-5L has been included within the national community stroke clinical audit (eg, SSNAP),50 which may enable longer-term and comparative stroke patient outcome review within future evaluation. Notably, staff valued NROL for increasing therapy and practice. Quantitative data supported their views, showing patients with stroke who participated in NROL received over four times as many therapy minutes as non-NROL patients and were supported by community services for longer, marking steps toward meeting therapy targets.

Workforce limitations remain a major barrier to implementation and sustainability in healthcare systems.45 51 We employed collective use of the regional rehabilitation therapy workforce, as a shared care approach, to help mitigate this challenge. This approach yielded several advantages. First, it allowed a critical mass of patients to receive group therapy where impairment incidence was lower (eg, speech-related groups). The cross-organisational structure (69% of sessions run cross-organisationally) reduced the need for multiple therapy staff from any one service per session, while providing opportunity for shared learning and networking. It also offered redundancy, allowing staff shortages in one service to be temporarily covered by another, minimising regional inequity. Although a concern regarding workload balance across organisations was voiced, staffing distribution aligned relatively well with referral patterns (variation −6% to 7%) suggesting a natural equilibrium or effective management. The collaborative system-level approach was enabled by the online format, which had the additional advantages of travel time, cost and emissions avoidance. Further, overall therapy staff resource use was minimised through the group-based model. Together, these advantages may help alleviate pressure on the healthcare system.

Wide stakeholder engagement fostered buy-in and short-term sustainment of the regional NROL innovation. Therapy staff and patients were integral to successful development and implementation as shown here and seen previously.19 27 However, regular communication with decision-makers was also crucial for scale-up, though this perspective is yet to be formally captured as it was beyond the resource of this evaluation. Alignment (ie, creating fit between elements of the inner and outer setting of an organisation or system) plays an important role for implementation success.52 53 Anecdotally, decision-makers found NROL appealing due to its strategic alignment with community-based care priorities, its low operational resource use and its contribution to broader healthcare goals, including workforce planning, performance benchmarks (eg, SSNAP), environmental targets and addressing health inequalities. However, sustained funding is essential to prevent ‘pilotitis’ and ensure long-term viability.

Some limitations should be considered when interpreting the findings. First, the therapy provision review was small-scale and limited to patients with a diagnosis of stroke, as clinical audit data were unavailable for other neurological conditions. Also, we only acquired access to stroke clinical audit datasets from two organisations. Further, interpretation should consider that the patients with stroke who participated in NROL may have been those who needed, could manage or were motivated for therapy, potentially affecting therapy amount and duration. Second, the staff survey had a 36% estimated response rate (55% respondents had delivered NROL), introducing potential non-response bias. This is not unexpected, as subject familiarity increases survey participation likelihood. However, it is important to note that those with less positive experience may be underrepresented. However, a strength of a mixed-methods approach is that the use of qualitative and quantitative data can help illustrate converging or diverging factors, providing a more comprehensive view. Finally, it was difficult to capture what was not done (eg, who was not referred). While overall, NROL had sustained appropriateness for mixed demographics, we found patients referred to NROL and who went on to participate were younger; this requires further exploration. Going forward, these perspectives will be sought, as well as the aforementioned perspective of decision-makers.

Conclusions

NROL was adopted and embedded into usual care at a regional level as an adjunct to enhance neurorehabilitation. It was accepted at a system level and increased the amount of therapy provided to patients. Workforce efficiencies were identified through effective collaborative systems working, with benefits amplified as participation grew. Longer-term sustainment of this regional innovation will require a compelling business case and value proposition for decision-makers. Ongoing work will address broader economic factors, focus on equality and improve operational efficiencies while supporting environmental goals. Importantly, adjustments to the model will be explored to optimise capacity and further intensify the therapy offer.

Supplementary material

10.1136/bmjopen-2025-101820online supplemental file 1

10.1136/bmjopen-2025-101820online supplemental file 2