Implementation-effectiveness of the power over pain portal for patients awaiting a tertiary care consultation for chronic pain: A pilot feasibility study

TL;DR

A digital platform called the Power Over Pain Portal was tested for its feasibility and effectiveness in helping people manage chronic pain before their first specialist appointment.

Contribution

This study evaluates the feasibility and acceptability of a digital chronic pain self-management tool in a tertiary care setting.

Findings

The recruitment and retention rates were 83.75% and 61.19%, indicating the study was feasible.

Participants showed reduced pain interference and belief in a medical cure after using the portal.

Users reported satisfaction and good usability, though some found the portal overwhelming.

Abstract

The Power Over Pain (POP) Portal is a digital platform that provides people living with pain (PLWP) flexible access to chronic pain self-management resources.

To (1) determine the feasibility of an adequately-powered multisite trial of the POP Portal in tertiary settings; (2) understand the acceptability and usability of the POP Portal; and (3) explore clinical effectiveness among PLWP awaiting a first visit to a tertiary care pain clinic.

Mixed-methods pilot-feasibility study to inform a future definitive trial. Feasibility was assessed using recruitment and retention rates. Acceptability, usability, and patient outcomes were measured using validated surveys completed at baseline and 3-month follow-up, and semistructured interviews conducted after 3-month follow-up.

Forty-one participants completed follow-up surveys and nine completed interviews. We reached a recruitment and retention rate of 83.75% and 61.19%, respectively. There was a reduction in pain interference (p = .024) and belief in a medical cure (p = .033) after using POP for 3 months. Surveys and interviews indicate PLWP were satisfied with the POP Portal, and it had good usability. Some participants indicated that POP was overwhelming, and certain resources were difficult to access, indicating that modifications could be made to improve ease of use.

The POP Portal was deemed acceptable with good usability; however, modification may be made for improvement. A definitive trial can proceed with enhancements to the portal, modification of the protocol, and close monitoring.

Figures

Core components of stepped care 2.0.

Flowchart of participant recruitment.

Full text

Chronic pain is described as pain, whether continuous or intermittent, which lasts for more than three months.^ 1 ^ Approximately 8 million children, youth, and adults live with chronic pain in Canada, and this number is anticipated to increase to 9 million by 2030.^ 2 ^ Chronic pain is estimated to cost over $40 billion per annum in direct (e.g., physician services, prescription medication, hospital inpatient and outpatient care) and indirect costs (e.g., time off work),^ 2 ^ accounts for approximately 10%–16% of visits to hospital emergency departments, and is experienced by the majority of frequent visitors of the emergency department.^3,4^

Chronic pain affects an individual's life physically, psychologically, socially, and financially. People living with pain (PLWP) experience reduced physical functioning over time and are less likely to return to work than people who do not experience pain on a daily basis.^ 5 ^ Without employment, PLWP are susceptible to earning a low income and experience financial burden.^6,7^ Additionally, an estimated 11% of PLWP experience a substance use disorder,^ 8 ^ and 5.5% of PLWP may develop an opioid use disorder when prescribed opioids for at least three months.^ 9 ^ Further, an estimated 60% of PLWP experience clinically relevant levels of depression or anxiety,^10,11^ and are often victims of stigmatization by family, friends, and health care professionals.^ 12 ^

In 2019, the Government of Canada formed the Canadian Pain Task Force to inform improved pain prevention and management efforts, focusing on six goals: (1) encouraging collaboration; (2) improving access to patient-centered pain care; (3) increasing awareness and training; (4) supporting innovation in pain research; (5) monitoring health system quality; and (6) development of equitable approaches to pain management.^ 2 ^ Since 2019, the Canadian Pain Task Force has engaged in consultations to identify best practices and gather suggestions for the development of strategies for pain management within Canada. These consultations revealed gaps in timely access to pain management resources, with treatment costs acting as a hurdle for low-income patients and resulting in inequitable and inadequate treatment for chronic pain.

Access to chronic pain care

Access to chronic pain care across Canada continues to be a challenge, with wait times for a first appointment ranging from a couple of months to several years.^ 13 ^ Approximately 50% of individuals waiting for an appointment are waitlisted for at least 6 months,^ 13 ^ which has been deemed to be medically unacceptable due to the adverse impact of waiting for care.^ 14 ^ Extended wait times have been found to coincide with worsening of pain and deterioration in physical function and quality of life.^14,15^

Digital eHealth platforms have shown promising results for improving access to care and patient outcomes. Slattery et al.^ 16 ^ conducted a systematic review of the literature reporting on the effectiveness of eHealth interventions among adults with chronic pain and identified 30 randomized control trials that reported on 5394 participants. eHealth interventions (e.g., mobile apps, telephone, virtual reality) were found to decrease pain interference, pain intensity, and psychological distress when compared to an enhanced control group (e.g., educational booklet). There has been a growing presence of chronic pain management apps^ 17 ^; however, none have provided: (1) resources that simultaneously address an individual's personal goals, needs, preferences, and readiness; (2) a platform that considers the relationship between chronic pain, mental health, and substance use; and (3) continuous outcome monitoring for treatment adjustments and evaluation.

Chronic pain care within a stepped care model

Stepped Care models incorporate healthcare services in a graded approach, ranging from low-intensity (e.g., initial assessment, psychoeducation), medium-intensity (e.g., medication and combined treatments), and high-intensity (e.g., in-patient care) services.^ 18 ^ Many programs implement a progressive model within their framework, whereby patients are first introduced to the lowest intensity service, and outcomes are reviewed to determine if the patient requires to be stepped up to the next level of care. Stepped Care models have previously been successfully implemented within the context of chronic pain care and have shown promise for improving access among PLWP. For instance, the US Veteran Affairs Stepped Care Model for Pain Management was implemented in 12 sites that serve approximately 130,000 patients and utilized a three-step approach that included primary care, pain specialists, and interdisciplinary care.^ 19 ^ Treatments followed a progressive model, whereby patients were first treated by a primary care provider. Treatment intensity is increased when needed. One study that sampled 25 healthcare providers and approximately 4000 patients observed an increase in pain knowledge and pain self-efficacy among patients, and improved pain care (e.g., adherence to treatment guidelines, documentation, and assessment effectiveness) after approximately 3 years of implementation.^ 20 ^

South Australia implemented a 3-step Stepped Care approach to pain management which incorporated population-level prevention and early intervention, primary care, and secondary/tertiary care.^ 19 ^ In this model, patients move through each step sequentially, meaning patients only receive tertiary care with a referral from a primary care provider. Despite the use of diverse Stepped Care models in chronic pain management, there remain no stratified approaches that consider a client's readiness and preferences in the treatment model.

Stepped care 2.0 at the Ottawa Hospital Pain Clinic

Stepped Care 2.0 is a model of care that reimagined the Stepped Care model developed in the UK^ 21 ^ and offers low- (e.g., education) to high-intensity (e.g., in-patient care) treatment options.^ 22 ^ Importantly, Stepped Care 2.0 contains nine core components to facilitate a client-centric approach to care (refer to Figure 1), whereby programs are co-designed with individuals of various perspectives to develop diverse levels of care that are recovery-oriented and improved through continuous monitoring and quality improvement cycles. Additionally, Stepped Care 2.0 promotes same-day access to person-centric care that is flexible, data-informed, and collaborative between clients and their healthcare providers. In contrast to other models, Stepped Care 2.0 offers flexibility across steps and offers clients autonomy to choose what resources match their needs, preferences, and readiness. Clients do so by working collaboratively with their healthcare provider to determine the best course of treatment.

Core components of stepped care 2.0.

The Ottawa Hospital (TOH) Pain Clinic adopted the Stepped Care 2.0 framework after transitioning from a multidisciplinary biopsychosocial approach to an interdisciplinary model of care to combat growing wait times of almost 6 months and to improve access to services and programs within the clinic.^ 23 ^ Patients referred to the Pain Clinic receive: (1) 90-min orientation; (2) medical visit; (3) interprofessional group education and intake session; and (4) 1:1 assessment for a personalized treatment plan before collaboratively deciding on resources that span an eight-tiered framework for chronic pain management. Within their individual plan, patients receive access to a combination of interventions across the continuum of care that are the least intensive and most appropriate for their needs. The Pain Clinic adopts a stratified model (i.e., patients are matched to the least intensive resource likely to meet their needs regardless of intensity of service), and care is flexible to patients’ needs over time. TOH Pain Clinic has reduced their clinic's wait times as patients receive a first call two days after receiving a referral on average, with completion of the assessment process in one month.^ 23 ^

The Power Over Pain Portal

The Power Over Pain (POP) Portal is an online portal built upon the Stepped Care 2.0 model, bringing together evidence-informed pain self-management interventions, designed to complement intensive care offerings (e.g., tertiary care) and augment a continuum of care. This ranges from educational resources (e.g., articles, videos), self-directed courses (e.g., the Pain Course), interactive workshops, and peer support (e.g., Ontario Self-Management), as well as resources for mental health and substance use concerns. Users also have the option to complete self-assessments to track their health progress over time. The POP Portal is dynamic and regularly updated as research on virtual care interventions for chronic pain progresses.

The POP Portal is designed to provide equitable, recovery-oriented, person-centered access to resources for pain management for all Canadians. The POP Portal was developed to align with several core components of Stepped Care 2.0, a model of Stepped Care that provides clients with flexible access to care at different intensities that can be matched to the clients’ needs, preferences and readiness, allowing increases or decreases in the intensity of their treatment at any time.^ 22 ^ Specifically, the POP Portal incorporates: (1) the involvement of key stakeholders throughout development, whereby the POP Portal involves PLWP in all development and refinement processes through the incorporation of a Lived Experience Advisory Committee; (2) offering a variety of step levels; (3) continuous improvements, by gathering feedback from users of the Portal and iteratively revising; (4) recovery-oriented services, by offering a variety of resources for users to choose from and acknowledging that each individual has diverse needs; (5) same-day access to pain self-management resources via a readily-accessible website, free for anyone to avail of; and (6) client-centricity, where emphasis is placed on client readiness and autonomy by encouraging the selection of preferred interventions. By following a Stepped Care model, the POP Portal strives to increase accessibility of care to patients in a time of need, while also maintaining an esteemed standard of care in chronic pain management.

TOH Pain Clinic has implemented the POP Portal into its referral pathway to enhance its Stepped Care 2.0 framework. At the time this was written, a clinic clerk contacts patients who were referred to the Pain Clinic to inform them about the POP Portal. An orientation session is offered if the patient is interested in learning more about the Portal to provide an in-depth overview of key offerings and features. Offering the POP Portal to wait-listed patients provides access to free self-management resources until an appointment can be made in the clinic.

Implementation of digital health interventions for chronic pain

Empirically-supported digital health interventions often do not achieve their expected impact due to poor uptake and engagement.^24,25^ Studies have shown variable attrition rates for eHealth interventions for chronic disease management, with only 11%–35% of participants continuing the use of the eHealth intervention at follow-up.^26,27^ Reasons for such attrition rates are uncertain; however, contributing factors may include participant age, social status, literacy skills, available guidance, intervention cost, and ability to personalize the experience.^28,29^

While resources on the POP Portal are evidence-supported and ready for implementation, it is important to consider specific factors which may influence the uptake and engagement of the Portal among PLWP and consequently influence the level of benefit patients receive after using the Portal. Identifying potential barriers and facilitators of implementation and measuring the intervention's acceptability and usability among key stakeholders is a crucial first step in this process.

The present study

The present study is a pilot feasibility study of implementing the POP Portal for PLWP awaiting a first visit at TOH Pain Clinic. The primary objectives of this study were to: (1) determine the feasibility of conducting an adequately-powered multisite trial that would evaluate the implementation of the POP Portal in a tertiary setting; (2) understand the acceptability and usability of the POP Portal among PLWP, including facilitators and barriers to the POP Portal's implementation; and (3) quantify the potential impact of the POP Portal among PLWP.

Methodology

Study design

This is a hybrid implementation-effectiveness type III pilot study using a prospective cohort, mixed method design. Surveys were delivered pre- and post-intervention to understand perceptions of the POP Portal's usability and acceptability and measure patient-reported outcomes, and then triangulated with responses from interviews. Using this design, the study focuses on designing and testing implementation strategies identified while determining the POP Portal's effectiveness on relevant clinical outcomes using multiple methods.^30,31^ This design provides enhanced external validity and the ability to assess implementation data while collecting clinical effectiveness data.^ 32 ^ The clinical effectiveness of individual components of the POP Portal has been established, and preliminary evidence supports the need for assessing the effectiveness of the POP Portal as an entity itself (i.e., the collective of evidence-based components). This approach enables a more robust examination of the determinants and contextual factors influencing implementation, thus providing comprehensive evidence to inform implementation strategy development and assessment.^31?–33^

The POP Portal is being implemented into the Ottawa Hospital Pain Clinic in alignment with stages of the Active Implementation Frameworks.^ 34 ^ Having previously engaged in exploration (e.g., identifying where in practice the POP Portal could fit, assessing fit and feasibility of options, conducting discussion sessions with providers, and engaging stakeholders in implementation planning via participation in the implementation team), and installation (e.g., mapping of referral pathway within TOH Pain Clinic with the POP Portal incorporated, continued development of implementation plan with implementation team, provide orientation sessions for providers and patients), this study was conducted in the installation stage. Data was gathered to inform decision-making and support the initial implementation of the POP Portal within TOH Pain Clinic, specifically targeting: (1) feasibility of implementation within the clinic's referral pathway; (2) identification of barrier and facilitators of use among patients; and (3) acceptability and usability of the POP Portal.

This study is reported in accordance with the CONSORT-extension for non-random pilot and feasibility trial guidelines.^ 35 ^

Participants

Inclusion criteria

Adults (>18 years of age) who experience chronic pain (i.e., ongoing, persistent, or recurrent pain for more than 3 months^ 36 ^), were referred to the TOH Pain Clinic, and have agreed to be contacted for research were eligible if they: (1) were able to speak, read, and comprehend English or French well enough to understand the resources (e.g., courses, workshops, videos) on the POP Portal; (2) provided informed consent in either English or French; and (3) had access to a computer, phone, or tablet with internet.

Exclusion criteria

Exclusion criteria included: (1) self-reported barriers to the use of technology during the initial screening appointment; (2) cancer-related pain; (3) inability to complete the study questionnaires; and (4) self-disclosure of an unmanaged serious mental health disorder (e.g., schizophrenia, bipolar disorder) or active suicidal ideations during the screening call.

Recruitment

Participants were recruited from the TOH Pain Clinic in Ottawa, Ontario, Canada. Patients who were waitlisted at the clinic were screened for eligibility by pain clinic clerks. A clinic clerk called eligible patients referred to the pain clinic who were awaiting triage and invited them to an Introduction to Chronic Pain Management Education Session (i.e., an existing session of the Pain Clinic's clinical program). During the call, the clerk informed patients about the current study and inquired about interest in additional information. The clinical administrative staff obtained consent to share the name, phone number, and email address of patients who expressed interest in the study with the clinical research assistant (AZ). Interested patients were contacted by the research assistant to arrange an orientation session.

The POP Portal was presented during the Introduction to Chronic Pain Management Education Session and details were provided about the present study. Patients who agreed to be contacted were informed that they would receive a call from a member of the research team. Patients who initially did not indicate an interest in the study were provided with the phone number of the clinical research assistant (AZ) and were invited to call to indicate their interest if they wished to do so at a later date.

The clinical research assistant (AZ) obtained verbal informed consent (an approved form of consent by The Ottawa Hospital Research Institute) or documented refusal to participate. A time was coordinated to complete baseline measures with those who consented to participate.

Participants were offered gift card(s) as an incentive for their participation. Those who completed the 3-month follow-ups were given a $20 Amazon gift card, and those who completed an interview received an additional$30 Amazon gift card. The study protocol was approved by The Ottawa Health Science Network Research Ethics Board (#20220443-01H).

Sample size determination

An a priori decision was made to recruit 80 patients from the TOH Pain Clinic triage pool or waitlist given that a minimum of 60 participants are recommended for pilot studies using continuous variables.^ 37 ^ Moreover, we intended to invite 12–15 of these participants to complete semistructured interviews in order to achieve saturation of information collected.^ 38 ^ Purposive sampling was used to ensure participants varied based on age, sex, and global impression of change over the 3-month study period.

Procedure

Consenting participants were asked to complete questionnaires using LimeSurvey at two timepoints: baseline and 3-month follow-up (refer to Table 1). After completing baseline questionnaires, participants were encouraged to use the POP Portal as they deemed appropriate over the subsequent 3-month period, including accessing resources on the portal, and completing self-assessments. Participants were prompted via email to complete follow-up questionnaires on LimeSurvey at 3-months’ time. Participants received one reminder call and those who had not yet completed the survey received one additional email after 1 week. After the completion of follow-up surveys, a subset of participants was invited via email to complete an interview at their earliest convenience (i.e., within two weeks) to further assess the acceptability, usability, and impact of the POP Portal. Interviews were conducted using a secure videoconference service (i.e., MS Teams) and were no longer than 45 min in duration. Interviews were recorded and transcribed using the videoconference service.

Table 1.: Schedule of assessments.

Measures

Demographics. Demographic information was collected including age, sex/gender, first three digits of postal code, ethnic background, Canadian province/territory of residence, and employment status. Information was also collected about participants’ pain onset (e.g., accident), diagnosis (if known), and impact on different aspects of the patient's daily life.

Baseline assessments

Brief Pain Inventory (BPI): The BPI is a validated instrument used to evaluate pain severity and the degree to which pain interferes with daily functioning.^ 39 ^ It includes a body schema allowing participants to indicate where they feel pain in their body, along with two pain sub-scales: intensity and interference. The pain intensity subscale includes four pain intensity ratings using an 11-point numerical rating scale ranging from 0 (no pain) to 10 (worst imaginable pain) on current pain, average pain, worst pain, and least pain during the last 24 h. The pain interference subscale is comprised of seven items that ask patients to rate the degree to which pain interferes with general activity, mood, walking ability, normal work (including housework), relations with other people, sleep, and enjoyment of life using an 11-point numerical rating scale ranging from 0 (does not interfere) to 10 (completely interferes). Higher scores indicate greater pain intensity and interference.^ 39 ^

Pain Self-Efficacy Questionnaire—Short Form (PSEQ-2): The PSEQ-2 is a validated two-item short form of the PSEQ-10, the 10-item tool for assessing the patient's pain self-efficacy (i.e., capability to perform activities even when undergoing pain).^ 40 ^ The two items ask the patient to rate their confidence in doing activities on a 7-point numerical rating scale (0–6), with higher scores indicating greater confidence.^ 40 ^

Chronic Pain Coping Inventory (CPCI): The CPCI is a validated tool used to assess the patient's use of strategies to cope with chronic pain.^ 41 ^ This questionnaire includes eight items measuring the use of Guarding, Resting, Asking for Assistance, Relaxation, Task Persistence, Exercise/Stretch, Seeking and Coping Self-Statements (1 item per domain). It asks the patient to indicate the number of days (0–7) during the past week that they used each of the strategies to manage pain. Higher scores indicate greater use of chronic pain coping strategies.

Survey of Pain Attitudes (SOPA): The SOPA is a validated tool used to assess the patient's attitudes and beliefs about their chronic pain.^ 41 ^ This seven-item instrument includes items on Pain Control, Disability, Harm, Emotion, Medication, Solicitude, and Medical Cure. Patients are asked to indicate how much they agree or disagree with each statement on a 5-point numerical rating scale ranging from 0 (this is very untrue for me) to 4 (this is very true for me).^ 41 ^

Short Form (SF)-12 v2: This 12-item short form is a validated self-reported measurement tool for physical and mental health.^ 42 ^ The survey touches on eight domains including Physical Functioning, Role-Physical, Bodily Pain, General Health, Vitality, Social Functioning, Role-Emotional, and Mental Health.^ 42 ^ Only the two-item physical functioning subscale of this tool is used in our study which evaluates limitations to physical activities due to current health. Response options include: (2) a lot of limitation, (1) little limitation, or (0) no limitation at all. Higher scores indicate greater limitations.

Generalized Anxiety Disorder-7 (GAD-7): GAD-7 is a validated seven-item instrument that evaluates symptoms associated with anxiety and the severity of generalized anxiety over the past 2 weeks.^ 43 ^ Patients are asked to endorse the number of days in which they have been bothered by symptoms using the following response options: not at all, several days, more than half the days, or nearly every day. GAD-7 has shown good validity and reliability in chronic pain samples.^ 44 ^

Patient Health Questionnaire-8 (PHQ-8): The PHQ-8 is a validated eight-item instrument that asks the patient to self-report the frequency with which they have been bothered by symptoms of depression over the past two weeks using four response options: not at all, several days, more than half the days, or nearly every day.^ 45 ^ The PHQ-8 omits the item pertaining to thoughts of death that is included on the PHQ-9. The PHQ-8 has shown good construct validity and internal consistency reliability.^ 46 ^

Follow-up assessments. The follow-up assessments that occurred at 3 months included all baseline measures along with the following:

Acceptability E-Scale (AES): Acceptability of the POP Portal is an implementation outcome of this study.^ 47 ^ Adapted from the 10-item Post-Survey Patient Impression Form, the AES is a six-item measure of the patient's experience with computerized programs, including how easy and enjoyable the program was to use, how understandable were the questions, how helpful was completing the program, whether the patient liked the program, whether the amount of time to complete the program was acceptable, and overall satisfaction with the program.^ 48 ^ Responses are made using a 5-point numerical rating scale (1–5) with anchors of very negative (1) to very positive (5) evaluation of the program. A total score of 24 has been identified as the threshold for good acceptability; however, individual items may be analyzed to provide specific insight into the areas of strength or weakness within the program. The AES has proven to have good construct validity and reliability when tested among adults.^ 48 ^

System Usability Scale (SUS): SUS is a reliable tool with good internal consistency, test-retest reliability, and concurrent validity for measuring the usability of systems when compared to other measures of system usability (e.g., Usability Metric for User Experience).^ 49 ^ It is a 10-statement questionnaire measuring the usability of a program, highlighting the need for support, training, and program complexity. Responses follow a 5-point numerical rating scale (1–5), with 1 indicating strong disagreement and 5 indicating strong agreement. Total scores range from 0–100, with higher scores indicating greater usability.^ 50 ^ Specifically, a mean SUS score of 12.5 indicates “worst imaginable” usability, 20.3 as “awful,” 35.7 as “poor,” 50.9 as “fair,” 71.4 as “good,” 85.5 as “excellent,” and 90.9 as “best imaginable.”^ 49 ^

Patients’ Global Impression of Change (PGIC) Scale: PGIC evaluates the perceived effect of disease management.^ 51 ^ In this study, patients were asked to rate their overall impression of change (i.e., ability to cope with pain, their overall status, and their pain level) that using the POP Portal made 3 months after first accessing it. Ratings were made using a 7-point Likert scale, ranging from (1) “very much worse” to (7) “very much improved”. A score of 5 or above indicates an improvement in patients’ symptoms.

Semi-structured interviews

A guide for semistructured interviews was developed a priori to capture: (1) usability which was informed by the System Usability Questionnaire^ 50 ^; (2) barriers and facilitators which were informed by the Theoretical Domains Framework^ 52 ^; (3) acceptability which was informed by the Theoretical Framework of Acceptability^ 53 ^; and (4) patient outcomes which were developed in collaboration with the POP Portal Steering Committee using an iterative process. This committee included pain experts from across Canada, as well as members of the Lived Experience Advisory Committee. Refer to Appendix C for the interview guide.

Interviews were conducted by a female research assistant (AK) who had previous experience conducting semistructured interviews with PLWP. The research assistant received training and supervision from a registered psychologist (PP). The research assistant had no established relationship with participants prior to the interview. The research assistant introduced herself as a research assistant for The Ottawa Hospital who was interested in helping to improve the Power Over Pain Portal for future users.

Feasibility outcomes. Recruitment Rate: The rate of recruitment was quantified as the number of consenting participants out of the number of PLWP invited to participate in the study.

Retention Rate: The rate of retention was quantified as the number of participants who completed 3-month follow-up measures out of the number of total consenting participants.

Analysis

Feasibility interpretation

Following recommendations for feasibility studies,^ 31 ^ we developed a priori criteria on our primary feasibility outcomes to indicate whether progression to an adequately powered larger trial was feasible. The a priori criteria serve as an implementation outcome^ 47 ^ and are organized through a traffic light system:

Green: Continue without modifications; this was indicated if we could: (1) recruit a minimum of 80 adults at TOH Pain Clinic over 3 months, (2) achieve a minimum of 80% retention rate (i.e., participants completing the Pain Course resource delivered through the POP Portal and pre-post outcome measures), and (3) if the majority (≥70%) of participants deem the POP Portal to be acceptable for addressing some of their pain and associated health concerns as measured by study questionnaires, Portal self-assessments, and/or interviews.

Yellow: Continue but modify protocol with close monitoring; this was indicated if we recruited 40–79 adult participants over 3 months, achieved 50%–79% retention rate, and/or 50%–69% of participants found the POP Portal acceptable.

Red: Definitive trial not feasible; this would be indicated if we recruited less than 40 adult participants over 3 months, achieved less than 50% retention rate, and/or <50% of participants found the POP Portal acceptable.

Quantitative analysis

The mean and variance of study outcomes were reported using descriptive statistics (frequencies and proportions) to understand the range of plausible effects that are likely to be observed in a definitive trial.

Paired samples t-tests were conducted to identify preliminary change in patient-reported outcomes (i.e., BPI, SOPA, CPCI, PSEQ, PHQ-8, GAD-7, SF12-PF) from baseline to follow-up. Pain interference and pain intensity were the primary outcomes of interest given their importance in pain management and the degree to which content on the POP Portal targeted these outcomes. The alpha value for analyses evaluating these outcomes was set to .025 using Bonferroni correction to account for inflation in family-wise error. A critical alpha of .05 was adopted to evaluate change in secondary (e.g., physical functioning, mental health measures) and exploratory (e.g., pain self-efficacy, chronic pain coping strategies, pain attitudes) outcomes given that the primary purpose of this study was to understand the acceptability and feasibility of the POP Portal, rather than the identification of change over time.

Qualitative analysis

Transcripts were iteratively revised and coded by two researchers (AK, NK-V) using the descriptive coding method described by Saldaña.^ 54 ^ Each sentence was coded based on the theme. Related themes were grouped into overarching themes. A codebook was developed from emerging codes and themes, where responses were quantified for each code. The codebook was then reviewed by a person with lived experience (NZ-N) to ensure accuracy. Themes that focus on effectiveness, acceptability, and usability were triangulated with quantitative data within their respective objectives. Specifically, findings of quantitative data were compared to the relevant themes identified in interviews to observe the validity of data collected and to provide in-depth information about where certain quantitative scores may derive from.

Results

Objective 1: determining the feasibility of an adequately powered trial

We invited 114 patients to participate in the present study during a recruitment period that spanned from 25 March 2023 to 7 August 2023. Eighty patients enrolled in the study, 14 could not be reached, 16 declined to participate, and four expressed interest, but did not enroll. The recruitment period was extended an additional 1.5 months to account for a relatively slow referral period to TOH Pain Clinic during the initial 3 months of recruitment, averaging approximately 170 referrals to TOH Pain Clinic each month.

Out of 80 enrolled, baseline surveys were completed by 67 patients, of which six cases were removed for completing <50% of the survey resulting in a sample of 61 patients at baseline. Follow-up surveys were completed by 41 participants, refer to Figure 2 for a depiction of patient flow throughout the study.

Flowchart of participant recruitment.

In total, 50 out of 80 patients who enrolled in the study availed of an orientation session. Of note, there was a change in technology partner which resulted in a revision to the POP Portal that occurred within the study period (i.e., a revised iteration of the POP Portal was relaunched on July 24, 2023). This revision included a new landing page, improved organization of resources, and visual improvements. Forty-five of the 50 patients received an orientation session to the original portal with only five patients using the revised portal from orientation throughout the study duration. Similarly, only three patients completed follow-up questionnaires before the portal was revised (i.e., most participants experienced a revision to Power Over Pain during their 3-month study period).

The majority of the 61 participants who completed >50% of baseline surveys were women (N = 40; 65.6%), North American (N = 34; 55.7%), and residing in Ontario (100%), with an average age of 52.44 years (SD = 16.98). Most participants were employed full-time (N = 13; 21.3%), retired (N = 11; 18.0%), or receiving financial support through provincial disability payments as a result of chronic pain (N = 10; 16.4%). Participants had lived with pain for a mean of 12.37 years (SD = 11.46). Refer to Table 2 for a summary of demographic information.

Table 2.: Summary of study demographics.

Given that 61 participants were enrolled in the study in the first 3 months, a recruitment rate of 70.2%, and a retention rate of 51.3%, this study meets the “yellow light” feasibility criteria for progression to a definitive trial. As such, we will continue with a future trial with modifications made to the study protocol and close monitoring.

Objective 2: evaluating the acceptability and usability of the POP portal among PLWP

Refer to Tables 3 and 4 for a summary of survey results, and qualitative findings, respectively. Interviews were conducted with nine participants who had completed follow-up measures. Saturation was determined to be the point at which no new information arose across three consecutive interviews and was reached after conducting interviews with nine patients. Most participants were male (N = 6), approximately 52 years of age, had a mean PGIC score of 3.56 (SD = 1.33), and had been living with continuous pain (N = 8) for on average 11 years, refer to Table 2.

Table 3.: Descriptive statistics and change in outcomes from baseline to 3 months.

Table 4.: Summary of qualitative findings.

Initial expectations of the Power Over Pain Portal

Participants anticipated specific features before accessing the POP Portal, including: (1) the ability to interact one-on-one with healthcare professionals (Theme E1.1; N = 4):I expected too much in terms of hoping that there was a one on one like maybe I was able to actually talk with somebody. (P18)

There were a variety of topics in which participants were interested in engaging with on the POP Portal, such as mental health resources to improve their mood and ability to cope with pain (Theme E2.1; N = 5), and non-pharmacological alternatives (Theme E2.2; N = 4):Two years ago I would have said that the pain aspect of it was most important, and now I would say mental health is number 1. (P14)

Perceptions of the Power Over Pain Portal’s Usability

Participant ratings of the usability of the POP Portal fell within the “good” range (MSUS = 70.56, SD = 20.95). Consistent with this, most participants who completed an interview described the POP Portal as intuitive and easy to use (Theme U1.1; N = 4), although some identified the Portal as difficult to navigate in the beginning due to the layout of the website (Theme U1.2; N = 3):Things would have to be laid out just a bit more user friendly because it seemed to me like I was jumping everywhere and there wasn't a clear structure. Maybe have more of a glossary or a table of contents. (P18)

Participants identified the POP Portal as an easily accessible tool that can be used in a time of need for rapid access to pain management resources (Theme U2.1; N = 3):Some days you're just looking for something. You're struggling or whatever, and just [access the portal] for 5 minutes. (P2)

Additionally, participants specified that diverse self-management resources and information on how to access province-specific resources made the POP Portal especially useful for PLWP who do not have access to a healthcare provider (Theme U2.2; N = 3):All the information that I found, I didn't have to go out of my home to find it. I didn't have to go to the doctor, you can find things out on your own without having to go to a doctor. (P3)

Participants identified several factors that could improve the POP Portal's usability. This includes improving the visibility of resources, as some resources were difficult to find (Theme U3.2; N = 3) and improving guidance on the Portal to reduce feeling overwhelmed by the number of resources available (Theme U3.1; N = 4):One of the things about the portal is there's a lot of information on there. And it's almost borderline overwhelming. But it's a catch 22 because there's good information on there. (P20)

Acceptability of the Power Over Pain Portal

Participants reported satisfaction with the POP Portal (MAES = 21.68, SD = 5.07); however, the threshold of acceptability (i.e., an average score of 24)^ 48 ^ was not reached which suggests that modifications are important to consider. More specifically, the POP Portal was identified as easy to use (M = 3.90/5, SD = 1.11), understand (M = 4.10/5, SD = .90), and had an acceptable time commitment (M = 3.88/5, SD = 1.24). Patients found the POP Portal slightly helpful in describing their symptoms and quality of life (M = 3.10/5, SD = 1.28), and were neutral in regards to the overall enjoyment of the Portal (M = 3.08/5, SD = 1.12). Interviews provided more detail about the specific strengths and shortcomings of the POP Portal with respect to its acceptability. Many participants identified having favorable experiences when using the POP Portal, enjoyed the available features (Theme A1.1; N = 5), and would recommend the website to other PLWP (Theme A1.2; N = 4):It was awesome. It was really good over the last three months to go through this program and see for myself, just the little things … it helped me a lot and I really appreciated these last three months. (P2)

Participants found using the self-assessment tool to be helpful in tracking their progress and symptoms over time, and often helped them to adjust activities to manage flare-ups (Theme A2.1; N = 5):I liked having the tracking system…it was nice to see the last like 3 months of data, how things fluctuate or stayed the same or maybe one week was different than the other. You can kind of go back and say, “OK, well, what was different that week?” What did I do maybe different that week or what changed that week? So yeah, the graph was helpful. (P2)

Other participants appreciated how the POP Portal endorsed the relationship between pain symptoms and mental health (Theme A2.2; N = 3):The one good thing I will tell you is I never dealt with my mental health. When I went to the pain doctor it was almost get in, get what you need and get out. … I was happy to know that [the POP Portal] were interested in the mental aspect of it. (P14)

The POP Portal was described by participants as having diverse resources to target a variety of needs, whereby participants can choose from resources to match their level of readiness, preferences, and concerns (Theme A3.1; N = 5):The portal gives people the opportunity to identify the kind of education that works for them. So people can choose, pick. You can choose them all and use everything, or you can use only what works for you and the way that you learn. So I think that it's doing that really well right now. (P12)

Many participants found the educational videos (Theme A3.2; N = 6) and articles (Theme A3.3; N = 3) located on the PoP Portal particularly helpful in building their understanding of pain, and felt that they provided them with “tips and tricks” to navigating pain self-management:I've watched a lot of the videos [on the portal], I'll be honest with you. Some of the videos of people talking about how to deal with pain and stuff like that. I thought, a lot of them were quite interesting. (P20)

Some participants specifically noted that they did not find resources specific to their diagnosis or concerns (Theme A4.1; N = 3):I didn't know what to do with the portal so I would just go in and click on some [resources], read it and then say OK there's a video by Doctor so and so then I click on that and go to it. Sometimes, honestly, I felt that it wasn't for me in terms of it didn't address what I was going through. (P18)

Suggestions to improve the acceptability of the POP Portal among PLWP, included: (1) incorporating additional one-on-one services with healthcare professionals who specialize in pain management (Theme A5.1; N = 6); (2) a feature to ask pain-related questions and receive prompt feedback (Theme A5.2; N = 3); (3) additional informational resources (e.g., financial support, medication use; Theme A5.3; N = 3):I think there needs to be someone at the other end of it right out of the gate that is a professional who knows what they're talking about, that's looking at my file and goes “OK, [participant]. I see what's going on with you. Let me explain it to you” and then direct me to the parts of the portal that can actually have concrete results. (P20)

Taken together, data from surveys and interviews suggest that resources available on the Portal at the time of this study were acceptable to participants; but with gaps that would be beneficial to fill (e.g., diagnosis-specific resources, one-on-one services with healthcare providers, financial resources). This suggests that incorporating such resources into the next iteration of the POP Portal would help to increase the Portal's acceptability beyond the threshold score indicating “good acceptability.”

Barriers of use

Participants reported barriers that could impact their use of the POP Portal. Some participants mentioned that those who are not knowledgeable of technology may have more difficulty accessing digital resources, such as those offered by the POP Portal (Theme B1.1; N = 3). Other participants indicated not engaging with resources, such as mental health resources, as they already receive care from a healthcare professional for the particular concern (Theme B2.1; N = 3):[Referring to mental health resources] I never used any of those resources because I'm already seeing a therapist for that. (P18)There were no facilitators of use identified by participants.

Objective 3: exploring the clinical effectiveness of the POP portal for PLWP

Participants reported improved outcomes across diverse indicators following the use of the Pain Portal. Specifically, improvements were noted in mental health (Theme O1.1; N = 4):I know that this is more of a chronic pain portal, but I will say that it helped me a lot with my psyche too (P14)

Further, reading about experiences from other PLWP was reported to facilitate the development of pain self-management strategies (Theme O1.2; N = 4):Just understanding someone else's perspective. In the general scheme of things, it does help to understand that people are going through the same thing that you are. And again, the tips and tricks that some of these people revealed was like, ohh, that's a good idea. Or, you know, I've tried that and it didn't work for me. (P11)

Furthermore, participants reported that engagement with the Portal improved their overall understanding of chronic pain (Theme O1.3; N = 4):It's really helping me to re-evaluate. I didn't think that in a month I would be taking more stock of myself. You know, being mindful of many different things. At the same time trying to live a normal life. It is key and not being 24/7 consumed by the pain that you're experiencing. Unfortunately, pain does dictate your life choices now, but it doesn't necessarily mean curling up in a ball and saying life is finished. No, not even close to that. So there's just constant reminders embedded within the content itself. (P11)

While not meeting the threshold of clinical significance, there was a significant reduction in pain-related interference from baseline to follow-up (Mdiff = −.66, SD = 1.78; refer to Table 3), indicating improved ability to engage with daily activities (e.g., walking, work, sleep) despite pain, t(39) = 2.357, p = .024, with a small to medium effect size of d = .373. Further, a significant decrease from baseline to follow-up was observed in the belief that a medical cure was necessary to improve pain (Mdiff = −.49, SD = 1.42), t(40) = 2.205, p = .033, d = .344. Triangulating data from interviews and surveys, we hypothesize that content on the portal resonated with the participants who developed a better understanding of pain and its impact. Such an understanding likely shaped beliefs around the uptake of self-management relative to the need for a medical cure resulting in the adoption of self-management approaches that reduced the degree to which pain impacts important areas of one's life.

Of note, no significant reduction in pain intensity was observed from baseline to follow-up (Mdiff = −.18, SD = .98), t(39) = 1.125, p = .268, d = .178. This was consistent with findings from interviews, whereby some participants reported benefits while others reported that the POP Portal was not helpful with improving their pain intensity, and did not offer tangible solutions (Theme O2.1; N = 3):More so the pain symptoms. It helped me with knowing what I’m dealing with, that was pretty much it. (P4)

No significant change was observed in physical functioning, PHQ-8, GAD-7, pain self-efficacy, or chronic pain coping strategies from baseline to follow-up.

Discussion

The present study set out to determine the feasibility of conducting an adequately powered trial evaluating the implementation of the POP Portal within tertiary care pain clinics. We further evaluated the acceptability and usability, and explored the potential clinical effectiveness of the POP Portal among PLWP. Surveys were administered to participants before and after use of the POP Portal for three months. Interviews were conducted with a select sample of participants to gain additional insights. This study was determined to be feasible with modifications to the study protocol (e.g., extending the recruitment period, adjusting the required sample size based on retention rates) and close monitoring. Participants endorsed the usability and acceptability of the POP Portal in part due to its varied resources to meet diverse needs. Importantly, improvements to the POP Portal were identified to increase acceptability and usability among PLWP. Our findings also showed mixed evidence of the POP Portal's impact on individuals' lives while living with pain by delivering strategies to improve understanding of pain, self-efficacy and confidence.

Patient outcomes

No reliable change was observed in patient's pain intensity during 3 months access to the portal; however, there was a reliable change in pain-related interference. It is important to note that this change did not meet the threshold for a clinically important improvement.^ 55 ^ There are several possible explanations for this pattern of results. First, participants were observed to engage in more activities in their day-to-day lives despite pain which may have offset potential improvements in pain intensity. Second, surveys indicated that participants felt their conditions had deteriorated slightly during the three months when they had access to the POP Portal. Deterioration is common while awaiting tertiary pain care, of which many participants in this study had yet to receive a first visit with a tertiary care pain clinic or are awaiting surgery. Further, participants have been living with refractory chronic pain, which is resistant to change, for approximately twelve years. Fourth, there was no measure of participant engagement with the POP Portal and it is unclear what resources PLWP used, how frequently they availed of the Portal, or how long they spent on each resource. As such, it is possible that engagement with evidence-based resources could have been low, resulting in little improvement in patient outcomes. Finally, participants reported that many of the resources on the POP Portal highlighted content and strategies that they have already tried or learned during their time living with pain, and endorsed these resources to other PLWP and family members who were not as knowledgeable about pain management. It is important to note that participants perceived the POP Portal as a valuable resource for pain self-management regardless of whether they saw personal benefit.

One of the goals of the POP Portal is to provide a sense of community among PLWP. This is facilitated by incorporating peer support resources, articles discussing living with pain written by PLWP, and weekly workshops. Social isolation has been identified as a significant predictor of pain interference, whereby individuals who do not feel included or do not engage with others identify a greater impact of pain on their daily functioning.^ 56 ^ PLWP who experience loneliness and lower levels of pain self-efficacy have shown greater interest in digital interventions, highlighting preferences for education-based resources, pathways to health services, and peer support.^ 57 ^ Interviews with the participants illustrated the POP Portal had provided them with a sense of community and felt they were not alone. Additionally, surveys indicated a significant reduction in pain interference among participants. This may imply that the POP Portal could offer a welcoming environment for PLWP, reducing social isolation and pain interference.

Managing expectations and adapting to patient needs

Many participants indicated they had anticipated greater 1:1 interaction with a healthcare provider while using the POP Portal, and described feeling lost or overwhelmed by the amount of resources offered. Expectations of contact with a healthcare provider have been observed in a similar study which implemented a digital self-management tool among patients with chronic obstructive pulmonary disease.^ 58 ^ The study highlights that healthcare providers had difficulty motivating patients to engage in self-management, as they felt they needed human contact and personalized prescribing to use the digital self-management tool.^ 58 ^ Personalized activity prescribing and digital monitoring have previously been used for self-management of chronic diseases (e.g., diabetes), whereby healthcare professionals prescribe physical activity to patients who are then provided self-monitoring kits (e.g., blood pressure monitors, step counters, glucose monitors).^ 59 ^ Patients participating in this personalized activity prescribing for twelve weeks evidenced significant reductions in fasting glucose levels, indicating patients had benefited from prescription of physical activity for blood glucose self-management. Healthcare providers may integrate the POP Portal into practice using similar methods with patients, whereby specific resources may be collaboratively discussed with the patient to arrive at a care option through shared-care decision-making, targeting outcomes measured using self-assessment surveys provided on the POP Portal. Such an approach is consistent with measurement-based care which has been shown to independently result in improvements in patient-reported outcomes.^60,61^

Participants indicated that the POP Portal met their expectations of usability; however, there may still be room for improvement to make the portal meet the recommended thresholds for acceptability. Interviews with the participants highlighted the Portal as intuitive and available during a time of need, although some participants felt overwhelmed and found some resources difficult to locate. The capability, opportunity, and motivation model of behavior (COM-B) model explains the influence of each of these three factors on behavior change.^ 62 ^ An individual's capability, opportunity, and motivation to engage in a behavior are closely related and each can have a direct influence on another. In this case, some participants have indicated a lack of physical capability to engage with resources on the POP Portal (i.e., ability to locate specific resources), which may reduce their reflective motivation (e.g., commitment to the Portal, awareness that the Portal is inclusive to individuals who may not be as knowledgeable on the internet). It may be necessary to revise the POP Portal to create a more inclusive and accessible environment for a diverse community of users. This may include a more streamlined pathway to resources, an option to search items using a search bar, or an online support representative to guide users to resources they may be looking for.

Implementation of the POP Portal into routine care

The present study identified determinants detailed in the Consolidated Framework for Implementation Research (CFIR)^ 63 ^ that are likely to influence the implementation of the POP Portal into routine care at a tertiary pain clinic. Two determinants of note were identified within the innovation domain: (1) relative advantage (e.g., PLWP question the advantage of using a digital health resource if they are seeing a healthcare provider for the same concern); and (2) innovation design (e.g., while there were diverse resources available on the Portal, some users were overwhelmed trying to find the resource that they were looking for). One characteristic of note was identified within the individual domain: capabilities of the innovation recipients (e.g., confidence of PLWP in navigating the Portal and adapting use based on their needs).

We used the Expert Recommendations for Implementing Change (ERIC)-CFIR matching tool^ 64 ^ to identify two fit-for-purpose interventions targeting the aforementioned implementation determinants. Identifying and preparing champions was the strategy identified with the greatest likelihood of influencing the perceived relative advantage of the POP Portal, and capabilities of those who utilize the portal. Program champions foster social opportunities for behavior change, as identified by the COM-B model,^ 62 ^ by endorsing the use of the POP Portal through peer navigation and testimonials. Providers who are also champions would also benefit capabilities and confidence among care providers, facilitating a common understanding of the intervention among colleagues and how it may be applied within their organization.^ 65 ^ The use of program champions has been identified as a particularly beneficial strategy for the successful implementation of eHealth interventions for patients with chronic illnesses.^ 66 ^

The development of educational materials was the strategy identified with the greatest likelihood of influencing the innovation design. Development of educational materials provides innovation recipients guidance on how to engage with the innovation (e.g., training materials).^62,63^ Decision guides are evidence-informed resources that highlight potential resources for patient care, offering evidence on the potential benefits and harms of each option.^ 67 ^ As highlighted in interviews with patients, the POP Portal offers an overwhelming amount of resources that can leave patients uncertain of which resource to engage with. Some patients also noted that they would like to see a healthcare provider prescribe resources based on what would work best for their present condition. A patient decision guide may help alleviate these concerns, whereby users could utilize the patient decision guide to identify resources with the greatest benefits while considering the potential drawbacks, and allowing an informed choice from a tailored list of available resources.

Insufficient quality monitoring was a barrier to implementing the POP Portal within the clinic that was noted during informal consultation with clinic staff. A rigorous process to monitor fidelity to the implementation of the POP Portal within the clinic will be used to improve implementation (i.e., enhancement fidelity^ 68 ^) during the full trial by ensuring staff of TOH Pain Clinic are engaging patients with the POP Portal through the implemented referral pathway. Quality monitoring provides pertinent information about how an intervention is implemented, which can be used to iteratively refine how the intervention is used in practice and tailor the approach based on the setting's barriers and facilitators of implementation, allowing for continuous improvement.^ 69 ^

Limitations

The results of the present study must be taken into account with the following limitations. The sample size was relatively small and powered for feasibility outcomes (i.e., the study was not sufficiently powered to evaluate change in clinical outcomes over time, whereby this study is only powered to detect effects of d = .50 in magnitude given power = .8, α = .025, and N = 41). Future research may evaluate clinical effectiveness outcomes in an adequately powered trial. Additionally, while consent was provided to access data pertaining to user engagement (e.g., frequency and duration of access), hosting of the POP Portal underwent an update and was transitioned to a new technology partner during the study period which prevented the collection of reliable data pertaining to user engagement. Further, no data were collected on resource utilization such that it was unknown which resources were used. Therefore, it is difficult to ascertain whether participants engaged in resources that have demonstrated efficacy in improving processes such as pain self-efficacy, and outcomes such as pain intensity and pain-related interference. Also of note, neither willingness to engage in pain self-management nor digital literacy was measured. These factors could have directly influenced participants’ ability to engage with the Portal as an eHealth tool. Further, response bias was not measured to determine the likelihood of socially desirable responses. This may be of particular concern given that a self-selecting sample was used which increases the likelihood of response bias. Additionally, participants were included only if they had access to a device and internet connection. As such, results may not pertain to the small proportion of people referred for tertiary pain care who do not have a device or internet connection. Finally, measures of usability and acceptability were performed on the Portal as a whole and not on specific aspects of the Portal (e.g., courses, self-assessments, educational information, workshops) which precluded a granular understanding of POP and its associated features.

Future directions

Additional research is needed to identify factors that impact engagement with the POP Portal (i.e., who uses it, what resources are used, the frequency of use, and the components on the POP Portal which are most widely used), so that interventions may be developed to increase meaningful engagement. Future research would also benefit from identifying determinants that influence the implementation and uptake of the POP Portal among members of participating tertiary care pain clinics, and encouraging adoption within primary care clinics. With this in mind, future research could focus on the development of a behavioral intervention, such as a decision-support intervention, for healthcare professionals to encourage the adoption of the POP Portal in clinical practice. This would include identifying the specific needs of healthcare professionals by understanding barriers and facilitators to the adoption of the POP Portal in practice. By improving the integration of the POP Portal into clinical practice, there is an opportunity to increase the likelihood that PLWP receive access to evidence-based resources for pain self-management and engage in continuous progress monitoring.

Conclusions

A hybrid implementation-effectiveness type III pilot study was conducted to identify the feasibility of conducting an adequately-powered multisite trials, and determine the acceptability and usability of the POP Portal among PLWP. We found that the POP Portal has been helpful in growing patients’’ understanding of their pain and provided strategies for self-management while awaiting a first appointment at TOH pain clinic, although improvements should be made to enhance organization of the Portal. The present study met the “yellow light” criteria, meaning that we will proceed with a definitive trial following modifications made to the study protocol and close monitoring. Information collected in this study will also be used to enhance the POP Portal in an iterative process with additional investigation of patient engagement and feasibility of use within healthcare settings.

Supplemental Material

sj-docx-1-dhj-10.1177_20552076251326229 - Supplemental material for Implementation-effectiveness of the power over pain portal for patients awaiting a tertiary care consultation for chronic pain: A pilot feasibility studySupplemental material, sj-docx-1-dhj-10.1177_20552076251326229 for Implementation-effectiveness of the power over pain portal for patients awaiting a tertiary care consultation for chronic pain: A pilot feasibility study by Alesha C. King, Amin Zahrai, Etienne J. Bisson, Yaadwinder Shergill, Danielle Rice, Eugene Wai, Natalie Zur Nedden, Lynn Cooper, Daniel James, Joshua A. Rash, Rachael Bosma, Tim Ramsay and Patricia Poulin in DIGITAL HEALTH

sj-pdf-2-dhj-10.1177_20552076251326229 - Supplemental material for Implementation-effectiveness of the power over pain portal for patients awaiting a tertiary care consultation for chronic pain: A pilot feasibility studySupplemental material, sj-pdf-2-dhj-10.1177_20552076251326229 for Implementation-effectiveness of the power over pain portal for patients awaiting a tertiary care consultation for chronic pain: A pilot feasibility study by Alesha C. King, Amin Zahrai, Etienne J. Bisson, Yaadwinder Shergill, Danielle Rice, Eugene Wai, Natalie Zur Nedden, Lynn Cooper, Daniel James, Joshua A. Rash, Rachael Bosma, Tim Ramsay and Patricia Poulin in DIGITAL HEALTH