Enhancing infant pain assessment and treatment: investigating barriers, facilitators, and implementation outcomes with the ImPaC Resource

TL;DR

This study examines how to improve pain care for infants in NICUs using a web-based strategy called ImPaC, identifying factors that help or hinder its success.

Contribution

The study introduces a tailored implementation strategy for infant pain care and identifies specific factors influencing its adoption in NICUs.

Findings

Engaging key stakeholders and having champions were strongly associated with better implementation outcomes.

Available resources and relative priority were identified as major barriers to implementation.

Tailoring the strategy to local needs can improve the feasibility and fidelity of the ImPaC Resource.

Abstract

The Implementation of Infant Pain Practice Change (ImPaC) Resource is a 7-step, multifaceted, web-based implementation strategy to improve pain assessment and treatment in Neonatal Intensive Care Units (NICUs). We explored facilitators and barriers to implementing ImPaC and their relationship to implementation outcomes.

A hybrid type 1 effectiveness-implementation study was conducted using a cluster randomized controlled trial (reported elsewhere) and a mixed-method exploratory study design. Level 2 and 3 Canadian NICUs with >15 beds were invited to participate and were randomized to intervention (INT, n=12) or usual care (UC, n=11) groups. INT NICUs recruited a change team who accessed ImPaC for 6 months; UC NICUs were waitlisted for 6 months and then offered ImPaC. Focus groups were conducted with all change teams following ImPaC completion. The Consolidated Framework for Implementation Research (CFIR) guided interview questions and analyses. Professionally transcribed interview data were coded and analysed using directed content analysis. Valence (+/-) and strength (–2, –1, 0, +1, +2) were assigned for each CFIR construct/subconstruct. Inductive codes were identified. Relationships between CFIR constructs/subconstructs and ImPaC implementation outcomes (feasibility and fidelity) were determined.

83 NICU change team members (median 4/site) participated in focus groups; 1,105 discrete codes relating to 31 CFIR constructs/subconstructs were identified. The most frequent facilitator constructs were Design Quality and Packaging, Compatibility, Available Resources, Champions, Implementation Climate, and Engaging Key Stakeholders. Complexity and Reflecting and Evaluating were salient in 21 transcripts, and Patient Needs and Resources was identified in 20 NICUs. Available Resources and Relative Priority were barriers. A positive association existed between the feasibility of implementing ImPaC and Engaging Key Stakeholders (0.46, p=0.041), Champions (0.82, p=0.001), Relative Priority (0.75, p=0.001) and Networks and Communication (0.60, p=0.023). There was a positive relationship between Engaging Key Stakeholders (0.42, p=0.048), Relative Priority (0.85, p=0.002), Patient Needs and Resources (0.46, p=0.049) and Fidelity.

Site-specific tailoring to enhance facilitators (e.g., champions, implementation climate) and mitigate local barriers (e.g., resources, relative priority) will provide a viable influence on optimizing implementation outcomes.

The online version contains supplementary material available at 10.1186/s43058-026-00856-8.

Full text

Contributions to the literature

• There is a lack of evidence-based and accessible implementation strategies to improve pain practices across neonatal care settings.
• The Implementation of Infant Pain Practice Change (ImPaC) Resource effectively improved pain practices while being successfully implemented in a hybrid type 1 study.
• The most frequent facilitators (using CFIR) were Design Quality and Packaging, Compatibility, Available Resources, Champions, Implementation Climate, and Engaging Key Stakeholders. Available Resources and Relative Priority were barriers.
• Identifying barriers and facilitators to implementing ImPaC will enable tailoring the intervention and its implementation to the local level to promote clinical and implementation outcomes.

Background

Successful implementation of research evidence into practice is a well-documented challenge [1]. Pain in infants is primarily associated with necessary clinical procedures and is frequently undertreated, unrecognised, or poorly managed [2, 3]. Persistent and untreated pain in early life can result in significant and enduring consequences that extend into adulthood, including continued chronic pain, disability, and distress [2]. This undertreatment of pain need not continue given the available tools, expertise, and evidence to prevent and treat pain in infants [2]. Educational materials, readiness for change questionnaires, audits, and intervention or quality improvement (QI) templates are implementation tools that have been reported to effectively change outcomes in single hospital Neonatal Intensive Care Units (NICUs) [4–8]. However, we are unaware of any comprehensive and accessible implementation strategies to improve pain assessment and treatment more globally across infants and settings.

The Implementation of Infant Pain Practice Change (ImPaC*) Resource* (Table 1) is a 7-step, multifaceted, web-based implementation strategy designed for healthcare providers to improve assessment and treatment of procedural pain in NICUs. ImPaC is comprised of strategies such as champions (Table 1, Step1), readiness assessment (Table 1, Step 2), audit and feedback (Table 1, Steps 3 and 6), a library of up-to-date and evidence-based knowledge translation tools on pain assessment and management (Table 1, Step 4) that enables champions to foster a collaborative learning environment while implementing these tools in their clinical settings (Table 1, Step 5), and finally a plan for next steps (Table 1, Step 7). ImPaC builds on the “Evidence-based Practice for Improving Quality” (EPIQ) for improving pain practices in children, which is comprised of high-quality evidence and QI methods using interactive implementation strategies. Improved pain practices and clinical outcomes were achieved in 16 Canadian hospital units that received the EPIQ intervention (INT) compared to 16 units that continued with usual care (UC) [10]. However, EPIQ was not web-based and thus was not always considered feasible or cost-effective [11], and results were only partially sustained over 12–36 months [10, 12]. ImPaC was developed using a user-centered process to address identified shortcomings and was deemed feasible, easy to navigate, engaging, intuitive, and comprehensive from a series of usability tests [13]. Table 1. ImPaC Resource steps [9]StepDescription of ImPaC StepsEstimated Time to Complete1Complete a checklist on the characteristics and strengths of a small team of 3–5 healthcare professionals responsible for implementing a practice change on the unit.4 weeks2Complete the ImPaC readiness for change survey and review your individual or team results.3Using the brief audit tool, perform about 10 infant chart audits to identify the unit’s current pain assessment and treatment practices.4a. Based on the audit results, select a pain assessment measure or pain treatment intervention that will be the focus for practice change.b. Review the evidence briefs. Consider measures or interventions that are the most feasible, relevant, and important for the infants, staff, and organization at this time.c. Create an aim statement and specify the percentage of change the team is aiming to achieve, and the time frame to achieve it. For example: We aim to increase pain assessment with the NIPs tool from 20% to 40% over the next 8 weeks.2–3 months per cycle5a. Select the knowledge translation (implementation) tools (e.g., mini presentations, stickers, screen savers) to help implement the pain assessment or treatment change into practice.b. Complete the activity planner to develop a detailed implementation plan. 6As in Step 3, use the audit tool to perform at least 10 infant charts audits following the practice change implementation period. Compare the audit results with the results generated in Step 3 to see if there is improvement. 7Evaluate the pain practice change and the implementation process. Repeat Steps 4–7 (cycle) until the desired practice change is achieved.

In a hybrid type 1 effectiveness-implementation study, 23 level 2 and 3 NICUs across Canada were enrolled in a cluster randomized controlled trial (cRCT) and randomized to the INT or UC groups [9]. In the INT group, the average number of procedures/infant/day was less, the proportion of procedures associated with pain assessment and pain treatment was greater, and pain intensity scores were lower. Although clinical outcomes were significantly improved, only 14/23 (60.9%) NICUs implemented ImPaC as intended for at least one complete 7-step cycle. NICUs spent an average of 10.18 (±4.36) hours implementing ImPaC over six months. More time spent using ImPaC with fidelity was associated with fewer painful procedures/infants/days and improved pain assessment and treatment strategies. Although the INT group outcomes were significantly better than the UC group, the pain assessment and treatment provision were still relatively low [9].

ImPaC is a multifaceted, novel implementation strategy that has the potential to bridge the knowledge-to-practice gap at the individual (e.g., by introducing interactive education and QI interventions and templates) and system (e.g., by promoting the use of champions/change teams) levels while determining change readiness and fostering a positive unit culture.

Understanding implementation barriers or facilitators and how they influence implementation outcomes is crucial to ensuring meaningful and sustainable pain practices.

Theoretical framework

The Consolidated Framework for Implementation Research (CFIR 1.0) [14] provided the conceptual basis for the study. CFIR is an implementation science (IS) determinant framework comprised of 39 constructs/subconstructs across five domains: (1) Intervention (Innovation) Characteristics, (2) Inner Setting, (3) Outer Setting, (4) Characteristics of Individuals, and (5) Process. CFIR guided the development of interview questions, coding and analyses.

Methods

Aim

We aimed to: (1) identify barriers and facilitators to implementing ImPaC in Canadian NICUs; and (2) explore the relationship between determinants and implementation outcomes (feasibility and fidelity).

Design and setting

This mixed-methods exploratory study was nested in a hybrid type 1 effectiveness-implementation study and a cRCT [9, 15]. Level 2 or 3 NICUs in pediatric or general hospitals across Canada were invited to participate. NICUs were eligible if they: (1) had a minimum of 15 beds; and (2) agreed to be engaged in the study for up to 24 months. NICUs were randomized to INT (n=12) or UC (n=11) groups using a computer-generated random allocation sequence (randomize.net).

Participants

Three to five individuals in each participating NICU were invited to join the ImPaC change team (CT) if they were a healthcare professional (HCP), English speaking, had ≥3 years of NICU experience, had flexibility and time within their role to engage in the study, and had clinical leadership experience (e.g., advanced practice or clinical education role). A local research coordinator obtained research consent. All CT members were trained and given access to ImPaC for 6 months. The INT NICUs received access to ImPaC immediately after randomization, while the UC NICUs were waitlisted for 6 months and then offered access to ImPaC. More details on participants recruitment and training are described elsewhere [9, 15]. Upon completion of the intervention for both the INT and UC groups, CT members were invited to participate in focus group (FG) interviews.

Data collection

Researchers (MB, BS) with qualitative research expertise conducted virtual FG interviews following the completion of the ImPaC intervention. Participants had no prior relationship with the interviewers. Interviewers used a CFIR-informed, semi-structured interview guide. Interviews were digitally recorded, de-identified, transcribed verbatim by a professional transcriptionist, and reviewed for accuracy. Each transcript represented a FG interview with a single NICU. Implementation outcomes were operationalized as user activity captured from the “back-end” of ImPaC. Feasibility (the number of hours the CT used ImPaC within the NICU) was assessed as an indicator of ImPaC’s ease of use in practice. Intervention fidelity was assessed as the degree to which ImPaC was implemented as intended (e.g., completing the seven steps of the change cycle).

Data analysis

Qualitative analysis

We undertook a deductive-inductive hybrid thematic analysis approach [16], allowing for structured analysis of data with a well-established and rigorously conceptual framework – CFIR guided a directed (deductive) content analysis – and for codes generated from the data itself, through inductive analysis. Our deductive approach was modelled from previous work done with the CFIR framework [17] and is described in detail in Additional file 1.

First, two research coordinators (KP and MR) familiarized themselves with the CFIR framework, construct/subconstruct definitions, and inclusion and exclusion criteria.

Second, the research coordinators and an IS researcher (MB) reviewed the CFIR framework and used the CFIR Codebook template [18] to create a coding dictionary in MAXQDA. Code memos were added detailing definitions and inclusion and exclusion criteria. The code memos were further developed using an iterative process to document how the CFIR codebook was operationalized to the study context [19]. Five subconstructs were not considered relevant to the study. They were not included in the coding dictionary (i.e., Trialability, Peer Pressure, Organizational Incentives and Rewards, Formally Appointed Internal Implementation Leaders, and External Change Agents), leaving the coding dictionary with 36 CFIR constructs and subconstructs. Next, the three analysists worked together to code the two transcripts line-by-line. Following consensus, segments were coded using a single CFIR code. Segments were inductively coded whenever the content was considered relevant to the study but did not align with CFIR. Inductive codes were created based on group consensus and used independently or combined with a CFIR code. The two research coordinators independently coded a third and fourth transcript. Then, all analysts compared CFIR codes, discussed any disagreements and, by consensus, reached decisions while generating a final coded transcript. The remaining transcripts were independently coded by one of the analysts and then reviewed by the others.

Strength and valence ratings were assigned to each construct based on a deliberate consensus approach in each transcript. These ratings were based on the content of the coded segments in relation to the negative or positive influence of the construct and/or subconstruct in the implementation process [14, 17]. They reflected the valence (positive [+] or negative [-] influence) and strength (from −2 to +2) of coded constructs and subconstructs in each transcript.

Summary reports for each transcript listed the ratings for each CFIR construct with at least one supporting segment. When all constructs and/or subconstructs were rated, they were compared across summaries (i.e., across sites) to help ensure consistency [17].

Quantitative analysis

Quantitative metrics were objectively determined from ImPaC on time spent using the Resource and the number of cycles or steps completed. Means and standard deviations were used to summarize continuous implementation outcomes: (a) feasibility (time spent by the change team using ImPaC in practice setting, calculated through automatic capture of the website’s back-end data metrics as users signed in/out) and (b) fidelity (adherence to completing the change team’s progression through steps or cycles). Spearman’s correlation was used to assess the association between each construct and the feasibility and fidelity outcomes. A p-value of less than 0.05 was considered statistically significant.

Reporting guidelines

This manuscript was prepared following the Standards for Reporting Implementation Studies (StaRI) [20].

Results

Facilitators and barriers to implementing ImPaC

We enrolled 107 CT participants and conducted 23 FG with 83/107 individuals (median 4 participant/site, range 1 to 7) from all participating NICUs. Interviews were conducted between January 21, 2021 and December 2, 2022, and lasted 25–60 minutes (Mean, 45.2, Median 46.5).

Table 2 reports the demographic characteristics and self-reported pain practices of the 107 CT participants. Most were full-time staff nurses with bachelor’s degrees. They were generally very experienced, and all were women. They reported a wide range of frequency of pain assessment and treatment practices as well as barriers and facilitators to implementing the ImPaC Resource and more generally, pain practice change. Table 2. Demographic characteristics and self-reported pain practicesDemographics and pain practicesN (%)Role Staff Nurse35 (48.0) Charge Nurse8 (11.0) Nurse Educator12 (16.4) Nurse Practitioner5 (6.9) Clinical Nurse Specialist1 (1.4) Physician1 (1.4) Other11 (15.1)Employment Full-time54 (74.0) Part-time (0.5 FTE or more)17 (23.3) Part-time (Less than 0.5 FTE)2 (2.7)Gender Female73 (100.0)Education Diploma/Certificate15 (20.6) Bachelors Degree53 (72.6) Masters Degree14 (19.3)Experience (in years) in current role Less than 1 year5 (6.9) 1 year to 5 years25 (34.3) 6 years to 10 years7 (9.6) More than 10 years36 (49.3)Experience (in years) in neonatal/pediatric care 1 year to 5 years18 (24.7) 6 years to 10 years5 (6.9) More than 10 years50 (68.5)Frequency of pain assessments in their clinical practice Never9 (12.3) Rarely9 (12.3) Sometimes19 (26.0) Most of the time16 (21.9) Always20 (27.4)Frequency of pain treatment provision in their clinical practice Never1 (1.4) Rarely10 (13.7) Sometimes21 (28.8) Most of the time24 (32.9) Always17 (23.3)

The analysists coded all transcripts, and 1,105 discrete codes relating to 31 CFIR constructs and subconstructs were identified. The most frequent or salient CFIR constructs were Design Quality and Packaging and Compatibility, coded in all 23 transcripts, followed by Available Resources, Champions, Implementation Climate, and Engaging Key Stakeholders, emerging in 22 transcripts. Complexity and Reflecting and Evaluation were evident in 21 transcripts, whereas Patient Needs and Resources was salient in 20 transcripts. CFIR constructs Goals and Feedback, Learning Climate, Individual Identification with the Organization, Readiness for Implementation, and Engaging, were not evident in any transcripts.

Details on the frequency, valence and strength attributed to each of the 31 constructs coded are presented in Tables 3 and 4. Constructs included in the coding dictionary but not identified as a facilitator or barrier by at least one CT were omitted. Table 3. Frequency of facilitators and barriers from coded transcriptsCFIR construct/subconstruct*** (italic)Barrier (−1,−2)Facilitator (+1, +2)Mixed(X)Neutral(0)TotalInnovation Characteristics Innovation source03003 Evidence strength and quality1151118 Relative advantage22015 Adaptability0111315 Complexity7131021 Design and quality packaging1211023 Cost390416Inner Setting Structural characteristics32027 Networks and communication1101315 Culture13116 Implementation climate7113122 *Tension for change (climate)*04004 *Compatibility (climate)*1180423 *Relative priority (climate)*1400216 *Leadership engagement (Readiness)*4111218 *Available resources (Readiness)*1541222 *Access to knowledge and information (Readiness)*770014Outer Setting* Needs & resources of those served by the organization0151420 Cosmopolitanism10023 External policy and incentives01001Characteristics of individuals Knowledge and beliefs about the innovation251715 Self-efficacy01012 Individual stage of change02002 Other personal attributes06006Process Planning14139 Opinion leaders (Engaging)02002 Champions (Engaging)852722 Key stakeholders (Engaging)1131722 Innovation participants (Engaging)13026 Executing551617 Reflecting and evaluating1120821Inductive code COVID-191500217Table 4Constructs and subconstructs strength and valence, per site**Site**Domain, Construct/Subconstruct**1**234567891011121314151617181920212223****Innovation characteristics - 7/8Innovation sourceNCNCNCNCNCNCNCNCNCNC+2NCNC+2NCNCNC+1NCNCNCNCNCEvidence strength and qualityNC+1+10+2+1X+2+1*+2+2−1NC+2NCNC+2+1+1+1+1NC+1Relative advantage−1NCNCNC+1−1NCNCNC0NCNCNCNC+1NCNCNCNCNCNCNCNCAdaptabilityX+1+1+1NCNC+1+10+1NCNCNCNC+1NC+10+1+10+1NCComplexity−1X+1+1+1−2NC+2+1−1+1+1−1−1*+1NC+1+1+1+1*−1*−1*+1Design and quality packaging+1*+2+1+1+2+1*+1+2+1*+1+1+1*+1*+2+1*−1+1+1+1+2+1X+1Cost0NC+10+2−1NC0−1+1NCNC+10−1NC+1+1+10+1+10Outer setting - 3/4Needs & resources of those served by the organizationNC+1NC+1+1+1+1+1+20+2+10+1X0NC+1+10+1+1+1CosmopolitanismNCNCNCNCNC0NCNCNCNCNCNCNCNCNCNC−1NCNC0NCNCNCExternal policy and incentivesNCNCNCNCNCNCNCNC+1NCNCNCNCNCNCNCNCNCNCNCNCNCNCInner setting - 13/14Structural characteristicsNCNC−1NCNC+1NCNCNCNC+1NC0NCNC0NCNCNCNC−1NC−1Networks and communicationNC+1+1NC+1+10NCNC0+1NCNC+1+1NCNC+1−1+1+10XCultureNC+1NCNCNC+1NCNCNC+10NCNCNCXNCNCNCNCNC−1NCNCImplementation climate+1+2−1+1−1+1−10−1X+1−1−2−1XNC+1+1+1+2X+1+1Tension for change (climate)NCNCNCNCNC+1+1NCNC+1NCNCNCNCNCNCNCNC+1NCNCNCNCCompatibility (climate)0+2+10+2+1+2+1+10+1+1+1+1+1+1+1+1+1−1+20+1Relative priority (climate)0NCNC−2−2−1NC−1−2−1−1NC−1−1NC−2−1−1NC−10−1NCGoals and feedbackNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCLearning climateNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCReadiness for implementationNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCLeadership engagement (readiness)NC+1NC+1+1+1+1+1−1NC+2+10−1−1XNC0+1NC+1−1+1Available resources (readiness)−1+1+1−2X−2+2−20−10−1−1−2−1−2−1+1−1NC−1−2−1Access to knowledge and information (readiness)−1+1+1NCNC−1NCNC−1+1+1NC−1*+1+1−1NCNCNC−1−1+1NCCharacteristics of individuals - 5/5Knowledge and beliefs about the innovation+1NC00NC0NC+10NC+1NCNCNC−100+10X+1−1NCSelf-efficacyNCNCNCNCNCNCNCNCNCNC+1NCNCNCNCNCNCNCNCNCNCNC0Individual stage of changeNCNCNCNCNCNCNCNCNC+1NCNCNCNCNCNCNCNCNCNCNCNC+1Individual identification with organizationNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCOther personal attributesNCNCNCNCNCNC+1NCNCNCNCNC+1NC+1NCNCNCNCNC+1+1+1Process - 8/10PlanningNC0NC+1NCNCNC+1+1NCNC+1NC0NC−1NC0NCNCXNCNCEngagingNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCNCOpinion leadersNCNCNCNCNCNCNCNCNCNC+2NCNCNCNCNCNCNCNCNC+1NCNCChampionsNC+10−1−10−10−1X+1−1*−1+10−2X000+1−2+1Key stakeholdersNC+2+1000+1*+1+2X+2+1−2+2+1000+20+2+2+1Innovation participantsNCNCNC+1NCNCNCNCNC−1+1NCNC+2NCNCNCNCNCNC0NC0Executing0NC+10NCNC0−1−1+2+1−1−10NC00NC+1NCX−1+1Reflecting and evaluating0+1+1+10+1NC+1+1+100−1+10NC0+1+1+1+100Barrier - COVID−2−1−1−2−2−2−2−10−10−1−1−1−1−2NCNCNCNCNC−1−1NC Not coded

Intervention characteristics

The perceived excellence in how the ImPaC is bundled, presented, and assembled [Design Quality and Packaging] was among the most salient implementation determinants, predominantly indicated as a facilitator.“It was very functional. It was very easy for us to use, and like I said before, the information is straight and to the point, so there’s not a lot of extra wording that people would just skim over or not even read at all. It was very user-friendly.” (Site W)

The library of KT tools (e.g., evidence briefs, PowerPoint presentations, posters, stickers, lanyards, etc., in Step 5 of ImPaC) was considered user-friendly, easy to navigate, well-designed, and organized.“Going into an environment that was already incredibly overwhelming at this time, the resources we were able to put forth to the staff were simple, they were concise, they were easy to follow, and it was not another level of information overload.” (Site E)

Evidence Strength and Quality– the stakeholders’ perceived quality and validity of evidence supporting the belief that the intervention will have the desired outcomes - was also a facilitator for most of the CTs:“It’s a lot of incredibly valuable information available at our fingertips. There’s no need to reinvent the wheel. It’s information and research that’s done, so why not use it, and it’s available to us. I think that alone is incredibly beneficial to be able to roll out so much in-depth and precise information on something that we clearly are supportive and strive to improve on within our unit.” (Site E)

The Intervention Source was described as another facilitator, with CTs recognizing ImPaC as developed by a reputable, credible group “for nurses by nurses”. (Site K)

Although the majority of the NICUs did not perceive ImPaC as a complex intervention [Complexity], some CTs felt that transitioning from one step to the next was challenging, especially while completing the pain practice audits in Steps 3 and 6:“I think it set us up pretty well for the following steps. I wouldn’t say that there was anything else that was missing. Again like having to go through each step one by one was helpful to get a sense of what we’re doing now. Okay we’ve completed that and now it’s the next one. So that and everything leading up to the actual implementation was fine.” (Site H)

Participants stated that ImPaC could be adapted or tailored to meet the NICUs’ needs [Adaptability], such as training newly hired clinicians and staff from units that eventually cared for the infants (e.g., pediatric units). One participant mentioned that ImPaC could be adapted to other practice changes as “it gives a good outline and a step-by-step process of how to carry out a change” (Site C).

The Cost of implementing ImPaC was considered minimal (e.g., using hospital printers to print posters). Still, operating costs would need to be considered by CTs when implementing the KT tools in their setting. A small budget was provided for teams to carry out these activities in the research study:“And I think, too, knowing that we have the budget, we were able to choose printable pieces like the posters and the lanyards, and if we didn’t have that budget, we would have considered more electronic like the screensavers or emails or other presentations.” (Site F)

Relative Advantage, the perception of the advantage of implementing the intervention versus an alternative solution, was a facilitator. CTs provided the team with “a solid place to go from” (Site E) and assisted with keeping up with timelines and the next steps. Conversely, others stated that ImPaC did not provide advantages to previous practice changes. Sites were comfortable using an externally developed tool for practice change in their units.

Inner setting

Compatibility (the fit between meaning and values attached to the intervention by involved individuals, how those align with individuals’ norms, values, and perceived risks and needs, and how the intervention fits with existing workflows and systems) was a facilitator.“I think that the tools actually were a reinforcement of the…what we have…the education that we have regarding the sucrose and it was just a really good reminder and it's nice to have some other education other than our voices for the staff to have something other than us to present what is out there.” (Site S)

The CTs described a favourable Implementation Climate characterized by receptiveness, interest and responsiveness to the innovation and the implementation process. A positive implementation climate was considered a facilitator by most with staff “(…) coming and saying can you tell me more about it.” (Site Q)“So in our unit it is very high acuity and very kind of fast paced…staffed with those exact personalities. And so I think everyone is really game for implementing change and very quick to adapt. So in that sense it was very feasible and I would say that to me it not only speaks to the tool but the people that I work with.” (Site T)

Some units indicated that “(…) people are just getting really burnt out with seeing here’s another change.” (Site M). In addition, the staff perceived that there were “(…) more databases to fill out, more paperwork to fill out” (Site I) because of the need to improve documentation of pain practices to assess practice change.

The support of mid-level leaders (e.g., clinical leaders and educators) was also evident [Leadership Engagement]. Several teams highlighted their commitment, involvement, and accountability as facilitative. Participants indicated that a “big change” (Site G) would not be possible without leadership engagement and stressed the importance of their active involvement with the CT members in discussing and making decisions on their implementation processes.“Yes, I definitely agree that having that leadership and upper management support is vital in terms of implementing the resource or really any practice change amongst the unit.” (Site K)

Local formal and informal Networks and Communications facilitated ImPaC within and across boundaries in the inner setting. Huddles, educational sessions, and educational boards as regular practices in the units favoured the implementation activities:“We discussed it every morning, too, in the morning huddle. So every week we would have new staff…residents, NPs, students, med students and, so then they would become aware of it and what our aim was and what our goals were.” (Site U)

Some teams indicated that they worked with IT departments to adjust their medical charting system and included relevant information for their practice change, highlighting the importance of communication for the CT's functioning.

Although all participants interacted with ImPaC and reported completing at least some of the seven core steps, Relative Priority was a barrier for some. One of the teams acknowledged that “(…) there always seems to be competing priorities and I don’t know if you will ever get to the point where there is no competing priorities. I think that’s just the nature of our environment.” (Site U) The COVID-19 pandemic, other quality improvement initiatives, implementation of electronic charting systems, staff turnover, staff shortages, and seasonal increased hospitalizations were some of the competing priorities mentioned:“But I think you also caught us in a time when we were also kind of getting…the Covid piece was slowly sorting itself out and now obviously there’s other pressures within the hospital in terms of pediatric viral illnesses that we’re definitely seeing trickling down to the NICU so I would say we’re just during a time of…I don’t want to say, like, high stress but lots of things happening, and I think, yes, it was feasible but perhaps wasn’t a super high priority for some of the bedside frontline staff. But I don’t think it impacted their ability to implement some of the tools.” (Site T)

The Available Resources dedicated to implementation and ongoing operations, including money, training, education, physical space, and time, were considered insufficient for ImPaC implementation in many sites, especially concerning time to work on their practice change.“We were understaffed. My assignments were so heavy, I barely got to my breaks. I’m barely getting to my breaks at all. So, just even having five minutes to kind of come back and look it was just…it was too much and it wasn’t a priority.” (Site P)

Outer setting

Within the outer setting domain, the most salient facilitator was the needs of the neonates and their families [Patient Needs and Resources]. Participants indicated that ImPaC brought awareness about pain assessment and treatment, making neonates and their families more comfortable in the NICU.“And people can get creative and realize that it is true…when it makes everyone’s…you feel your job easier, the baby is more comfortable, the parents are more comfortable…things will be more smooth and people will start to, maybe, be more creative and have more ideas of implementing the skin-to-skin or the breastfeeding but this is the kind of first step I think into that.” (Site K)

Characteristics of individuals

Participants considered themselves familiar with neonatal and infant pain knowledge and QI practices. Their Knowledge and Beliefs facilitated their role in the implementation of ImPaC:“Because a lot of us as people with leadership experience from different disciplines and many of us were on pain committee to begin with were used to these PDSA cycle and with the understanding of our group and what new practice implementation looks like…so I think we just dove headfirst into it.” (Site H)

Other Personal Attributes such as diversity in personal background, roles (e.g., clinicians, educators, NPs), clinical and technological expertise, and knowledge perspective contributed to implementation:“Yes I think it was helpful to have a variety of people with different backgrounds and different experience levels and what not.” (Site M)

Process

In several interviews, the staff members engaged in implementation activities described the CTs as facilitators [Engaging Key Stakeholders]. CTs used different strategies to engage staff, such as educational sessions, huddles, electronic reminders and “spontaneous discussions that happened at the bedside as well amongst the staff.” (Site B) Participants perceived that staff were receptive to the practice change, disseminating and implementing what they learned from the CTs at the bedside.

Reflecting and Evaluating on implementation progress and quality was also a facilitator. The audit tool and the pre- and post-intervention reports (Steps 3 and 6) enabled the CTs to reflect on their clinical and implementation outcomes and the results achieved and plan for their next steps. Some of the participants indicated they shared the results with staff, especially when their goals were achieved:“So I think people like to see numbers when they rise. So, showing them where we started at and what we are at now…people’s eyes were opened when we said this is what we achieved the last six months, and they are like, what? Wow. So I think people like that. And I like it, every now and then. So, I think if we have good numbers and we can show them that there’s been a change in the practice that has been reflected in our numbers, I think that would be a plus, every now and then.” (Site J)

Executing the implementation according to plan was considered a facilitator for CTs. ImPaC “(…) is quite reasonable. I was very impressed at how fast and we quickly moved to a positive outcome.” (Site S) However, other CTs “(…) didn’t get all the steps done. We got to probably step four or step five and then it just stopped.” (Site P) One site indicated the seven-step process was too long and was a barrier to implementing their practice change. Some CT champions felt overwhelmed trying to implement ImPaC:“Because we are a small group and it seems the same people are implementing the same projects, and so, it’s a bit of, also, overload, especially during Covid. So, I think you’re also/we need to mindful of how many champions we have when we are actually doing these types of programs because it’s almost overload for those that are rolling them out or ensuring that we are doing what we are supposed to be doing.” (Site V)

In addition, as Champions, CT members described how they needed to adapt their approach:“(…) We had to muster up energy. We…not only at the bedside but also with our colleagues. That was really difficult because we were coming with all this energy, but then for it not to be reciprocated was…we had to adapt. We had to recognize that, and I think that in and of itself was a huge challenge.” (Site G)

Relationship between implementation determinants and implementation outcomes

Several barriers and facilitators were associated with implementation outcomes (feasibility and fidelity) (Table 5). Table 5. Correlation between constructs and subconstructs and implementation outcomesConstruct/SubconstructCorrelation feasibilityp-valueCorrelation with fidelityp-valueInnovation CharacteristicsInnovation source--−0.980.12Evidence strength and quality0.060.810.040.86Relative advantage−0.500.400.99Adaptability−0.410.150.130.65Complexity−0.390.21−0.120.56Design and quality packaging0.350.120.250.26Cost−0.140.590.330.18Outer SettingPatient needs and resources0.190.460.460.049Cosmopolitanism0.950.210.990.084External policy and incentives----Inner SettingStructural characteristics−0.320.49−0.180.7Networks and communication0.600.0230.0040.99Culture−0.770.120.310.61Implementation climate0.200.430.150.63Tension for change**--**--Compatibility0.250.26−0.10.64Relative priority0.750.0010.850.002Goals and feedback----Learning climate----Readiness for implementation----Leadership engagement0.060.830.280.28Available resources0.230.330.040.86Access to knowledge and information0.270.36−0.90.75Characteristics of individualsKnowledge and beliefs about the innovation0.430.120.340.24Self-efficacy----Individual stage of change----Individual identification with organization----Other personal attributes---**-ProcessPlanning−0.020.970.130.75Engaging----Opinion leaders----Champions0.82<0.0010.130.59key stakeholders0.460.0410.420.048Innovation participants−0.100.85−0.630.18Executing0.310.270.260.32Reflecting and evaluating0.330.160.190.41Inductive codeCOVID-190.420.095−0.310.21Denotes statistically significant correlation

There was an association between implementation recipients and feasibility (0.46, p=0.041) and fidelity (0.42, p=0.048). There was also a significant relationship between champions and time using ImPaC (0.82, p=0.001).

CT members described how staff perceived ImPaC:“They were really receptive when we said it was just a quick minute and a half long video they were like okay…that’s something I’m willing to watch. So, I think that was really helpful and you know…I noticed lots of people watching it. I think that people…they didn’t feel like it was redundant or threatening.” (Site N)

Staff’s positive receptivity might have influenced the time CTs were able to use ImPaC in the NICU:“(…) I feel like that’s one of our most successful strategies, in terms of putting the information out there, and along with the videos and posters, we have some evidence briefings. Some of the…some of the evidence briefs from within the resource that we’ve printed and tried to make them readily accessible within the unit and in high traffic areas and like we had mentioned with the little competition that we’re just trying to raise awareness and promote and encourage everyone to document and continue doing these pain management…nonpharmacological pain management techniques.” (Site K)

Some CT members indicated that their role as Champions facilitated the implementation of ImPaC.I felt “(…) encouraged to be in a group so I didn’t have to take it on myself. So it just…I felt like the website encouraged that interaction as a team. You know? And using it.” (Site W) However, other CTs disagreed, because “(…) none of us really ever ran on the same schedule other than a couple of people. And…just finding time in the day to really sit down and go through anything was quite challenging”. (Site M)

Networks and Communications were significantly correlated with the feasibility of implementing ImPaC (0.60, p=0.023). CTs engaged in implementation activities through opportunities available at their unit, which mainly were perceived as facilitators:“We discussed it every morning too, in the morning huddle. So every week we would have new staff…residents, NPs, students, med students, and so, then they would become aware of it and what our aim was and what our goals were.” (Site U)

There was also a significant relationship between Relative Priority and feasibility (0.75, np=0.001) and fidelity (0.85, p=0.002). However, Relative Priority was also considered a barrier that often precluded them from spending time on the website.“I feel like everyone was already, like, super stressed about everything. They were stressed about COVID. They were stressed about EPIC. We happened to be probably the busiest we’ve been in a long time too during this period, as well.” (Site D)“And I think it was just like timing from our site. We were…we were extremely, extremely busy in the last few months, and we had management change, and we had staffing…serious staffing issues. So, then, just being at work was…is getting the work done” (Site I).

Finally, a significant relationship existed between Patient Needs and Resources and fidelity (0.46, p=0.049):“I think we’re very good for advocating for our patients and trying to ensure that they are comfortable. So staff bought into the value of it, so I think that made it more successful.” (Site I)

Discussion

Summary of findings and comparisons with other studies

This study aimed to explore barriers and facilitators of implementing ImPaC, a multifaceted novel implementation strategy, across 23 Canadian NICUs and to determine relationships between implementation determinants and implementation outcomes. The most frequent facilitators were within the Innovation Domain.

The iterative integrated implementation approach, bringing developers and users together, to develop and refine ImPaC resulted in its’ Design and Quality Packaging being identified as a facilitator by almost all the CTs. Also, using the web-based format allows for greater accessibility, relative ease to sustain and modify once developed, and automation and interactive capabilities [21].

In addition, the perceived Evidence Strength and Quality of ImPaC was highly salient and a key facilitator for practice change. Participants perceived the pain assessment and treatment tools as being of high quality and valid. The readily available library of resources on pain assessment and treatment comprised up-to-date, high-quality, synthesized evidence and was consistently mentioned by the CTs as a facilitator. Limited knowledge of pain assessment measures, approaches and treatment in neonates hospitalized in NICUs worldwide is consistently described as a barrier to the provision of optimal care by nurses [22–25]. Additionally, nurses and midwives indicate a lack of time to update knowledge and clinical practices related to pain [25]. Thus, readily available and valid implementation strategies are highly valued.

In the Inner Setting, Compatibility, or how the intervention fits existing workflows and systems, was a salient facilitator noted by the CTs. ImPaC relies on up-to-date, high-quality, synthesized infant pain assessment and treatment evidence, meeting Canadian and global standards [23, 24]. CTs implemented ImPaC without adaptations as it was compatible with local pain practices. Implementing ImPaC to meet infants’ needs [Patient Needs and Resources] on improved pain practices was another salient facilitator.

In a web-based survey distributed to NICUs worldwide, 67% of 303 NICUs indicated having local pain management guidelines [26]; however, research continues to demonstrate a significant knowledge to practice gap in procedural pain treatment [3, 27]. ImPaC may play an important role in minimizing this gap by focusing on the QI process (e.g., Plan, Do, Study, Act cycles) of implementing evidence-based information on infant pain at the unit level rather than simply providing evidence-based pain assessment and treatment strategies. This approach acknowledges a required paradigm shift in the responsibility for pain management to organizations rather than solely individuals. Organizations must acknowledge their institutional commitment towards pain prevention and treatment through accountability requirements in their organizational structure, prioritization of key outcomes, fundraising and strategic planning [28, 29].

Lack of Available Resources, predominantly time and competing Relative Priorities, were salient barriers identified by the CTs, with overlapping reasoning. NICUs are demanding and dynamic environments, and most individuals who joined the CTs did so in addition to rather than part of their current roles (e.g., educator, bedside nurse, advanced practice nursing roles). Some NICUs implemented ImPaC while transitioning from paper to electronic medical records, requiring significant training commitment. These resource issues raise the question of who is best suited for the CT role. Engaging individuals experienced in implementing QI strategies and managing multiple tasks in complex environments may be beneficial.

We also explored the relationships between implementation outcomes and determinants. We found several implementation determinants were associated with feasibility. These constructs included Champions, Key Stakeholders, Relative Priority and Networking and Communications. These positive associations with feasibility make logical sense in reflecting the availability and willingness to spend time interacting with the website and provide a starting place to tailor local strategies and develop future refinements to ImPaC to enhance uptake and implementation. We found fewer correlations between implementation determinants and intervention fidelity, including statistically significant associations between fidelity and Engaging Key Stakeholders, Relative Priority, and Patient Needs and Resources. Associations may be limited due to the small sample size of participating NICUs.

Strengths and limitations

The implementation process embedded in ImPaC steps was considered logical and easy to follow and navigate. Key implementation strategies such as identifying and preparing the CT (champions), assessing readiness, identifying local barriers and facilitators, audit and feedback, distributing educational materials, making training dynamic, reminding clinicians, and re-examining the implementation process [30], were considered helpful in guiding pain practice change and may be a template for other practice changes. The seven-step approach provided CTs a structured facilitative process for individuals unfamiliar with QI initiatives. ImPaC is based on years of research on pain in neonates, infants, and children [10, 12, 13, 15] with the collaboration of knowledge users and an interdisciplinary group of researchers. Users' collective expertise and consideration were central to CTs, as noted in Table 2. These attributes may have enabled some of the salient facilitators identified, such as Implementation Climate, Engaging Key Stakeholders, Reflecting and Evaluating, and Patient Needs and Resources. However, we expect that the user-centered design approach and series of usability testing which preceded this work, combined with key attributes of ImPaC were noted as facilitators, including Design Quality and Packaging and Complexity, allow for HCPs at any level of their career to be able to navigate through and implement ImPaC.

Fidelity was operationalized as adherence to completing all ImPaC’s steps. Although we consider the 7-step cycle progression as logical and theoretically based, participants indicated that sometimes they continued implementing the KT tools over time, without completing the final planning step (Step 7) or going back to ImPaC and formally completing earlier steps (e.g., Steps 3 to 6). This approach might indicate how ImPaC can be adapted to meet local needs; however, the impact on fidelity and sustainability over time and in maintaining clinical outcomes needs to be further investigated.

The COVID-19 pandemic was a commonly reported barrier to implementing ImPaC and the overall conduct of the study. This finding aligns with Critical Incidents, a new construct in the updated CFIR 2.0 framework [30] to include events such as pandemics, weather-related disasters, or political disruptions as ‘large-scale and/or unanticipated events disrupting innovation implementation and/or delivery. The COVID-19 pandemic interrupted research activities for about 3 months in 11 of the 23 participating sites, with many units needing to wait until the pandemic was over before starting implementation. In another setting in Italy, a survey conducted in seven level 3 and six level 2 NICUs revealed that despite the low direct clinical impact of COVID-19 in newborns, clinicians displayed clear signs of mental health load outcomes due to living in a situation of uncertainty and personal exposure to contacts with parents and other relatives of the newborns, and having to carry out activities once routine and now fraught with uncertainty [31].

Pain in preterm and sick infants hospitalized in the NICU is a global concern in terms of both immediate and long-term outcomes. Safe and effective interventions are needed to foster practice change on a larger scale. We are limited in using ImPaC in English-speaking NICUs in one high-income country; thus, cultural relevance to other settings that speak different languages, have different health care systems and are in various income brackets is unknown. We are currently updating ImPaC based on user input so that the tool is living and dynamic. In addition, we will address accessibility and cultural adaptation for a broader spectrum of NICUs. We will also consider the CFIR 2.0 version to include additional constructs and subconstructs for evaluation in the future, hoping to understand better determinants that can be tailored for better results.

Conclusion and implications

To our knowledge, ImPaC is the first multifaceted web-based implementation strategy designed to facilitate pain practice change in NICUs. We have created a web-based implementation strategy for HCPs caring for hospitalized infants. Our goal is to facilitate the implementation of evidence-based knowledge in a manner that allows for ease of practice change. NICU implementers of ImPaC have varying degrees of IS experience and understanding, requiring a simple and feasible process to guide the practice change work. Identifying barriers and facilitators to implementing ImPaC enables tailoring at the local level to promote clinical and implementation outcomes.

Supplementary Information

Additional file 1. Detailed criteria used to assign ratings to constructs.