Implementing a Screening, Brief Intervention, and Referral to Treatment Curriculum for Medical Students on their Emergency Department Rotation

TL;DR

This study developed and tested a curriculum to train medical students in SBIRT techniques during their emergency department rotation, showing improved attitudes and knowledge.

Contribution

A novel SBIRT curriculum for medical students was developed and demonstrated significant improvements in attitudes and preparedness.

Findings

Medical students showed significant improvements in attitudes and preparedness after the curriculum (p < .001).

Knowledge scores improved significantly (p = .002) and OSCE performance increased over three encounters (p = .03).

The curriculum effectively integrated SBIRT and motivational interviewing into medical education.

Abstract

The screening, brief intervention, and referral to treatment (SBIRT) approach is an evidence-based tool that combines standardized screening for unhealthy or risky alcohol and drug use with principles of motivational interviewing to promote behavior change and connect patients with the appropriate treatment and recovery support services. There is an increased demand for health care students and providers to be trained in SBIRT. We developed a curriculum to improve medical students’ attitudes toward and proficiency in administering SBIRT.

The curriculum was deployed as part of the emergency department clerkship of an undergraduate medical education program at an urban, safety net academic medical center. The content and structure, developed with input from medicine, nursing, and social work educators, consists of a 1-hour didactic session, three rounds of formative OSCE encounters, and one SBIRT delivery in the emergency department. Students were evaluated on their attitudes, sense of preparedness, and practical understanding of SBIRT.

Fifty-six medical students participated in the curriculum. There were significant differences between students’ pre- and postcurriculum attitudes and preparedness scores (p < .001) and knowledge scores (p = .002), and in OSCE scores between the first and third standardized patient encounter (p = .03).

This curriculum significantly impacted medical students’ attitudes and knowledge regarding SBIRT and motivational interviewing techniques. Widespread implementation of similar curricula could equip future physicians with the skills to implement evidence-based substance use screening and intervention into their practice.

Figures

Motivational interviewing (MI) techniques used during the screening, brief intervention, and referral to treatment (SBIRT) curriculum. Twenty students administered SBIRT to a patient, and 12 (60%) provided a narrative of their SBIRT interview, including MI techniques used.

Full text

Educational Objectives

By the end of this activity, learners will be able to:

• 1.Describe principles of motivational interviewing and screening, brief intervention, and referral to treatment (SBIRT), including risk stratification, normalization, addressing stigma, excessive drinking parameters, stages of change, open-ended questions, affirmations, reflections, and motivation-enhancing techniques.
• 2.Demonstrate improved attitudes and preparedness in medical students to administer SBIRT in clinical settings.
• 3.Practice administering motivational interviewing and SBIRT with standardized patients.
• 4.Apply techniques of motivational interviewing and SBIRT with a patient during their emergency medicine rotation.

Introduction

Screening, brief intervention, and referral to treatment (SBIRT) is an efficient and evidence-based approach grounded in motivational interviewing (MI) principles for implementing patient screening, providing educational materials, raising self-awareness, and enhancing motivation to engage in behavior change.^1^ There is substantial research demonstrating the effectiveness of SBIRT in reducing risky alcohol consumption, and accumulating evidence supporting the effectiveness of SBIRT in reducing risky drug use.^2–5^ The Centers for Medicare and Medicaid Services recognizes SBIRT as a recommended approach to systematically screen people who may not otherwise seek substance use help and offer treatment services that reduce health costs, drug and alcohol use severity, and the number of patients who go without specialized treatment.^6^ In the United States, there is a national push to develop curricula to train health care professionals, including medical residents and physicians, to become proficient in administering SBIRT.^7^ This has resulted in curriculum innovation, including SBIRT or MI training implementation into health professions education.^8–14^

OSCE and role-playing activities have previously been incorporated into successful approaches to health professions SBIRT and MI teaching.^9–11,13,14^ Student performance in tobacco cessation counseling and SBIRT was observed to be favorable following OSCE activities, standardized patient (SP) instruction, and participation in role-playing versus didactic or module-based approaches alone.^15^ Role-playing is an approach supported by the American Heart Association, given its effectiveness in preparing medical students for lifestyle counseling when combined with traditional didactic approaches.^16^ More generally, the OSCE has become the standard assessment method in medical education due to its effectiveness in evaluating key clinical skills through direct observation and in preparing medical students for clinical rotations.^17^

Our institution introduces MI techniques to first-year medical students (MS1s) by providing a 3-hour interactive lecture. While helpful, it is limited and does not cover SBIRT. To bridge these gaps, we developed an expanded SBIRT and MI curriculum for second-year medical students (MS2s) during their emergency department (ED) clerkship rotation. This curriculum aims to boost student readiness to administer SBIRT through a program of advanced training, practice with SPs involving real-time feedback, and encouragement to administer SBIRT on a patient in the ED.

Methods

We adopted the lecture materials and OSCE format materials developed by senior faculty in the Department of Family Medicine at Baylor College of Medicine and the Health Behavior Research and Training Institute at the University of Texas at Austin (UT Austin). We negotiated the retained and added components to this source material with faculty from the Steve Hicks School of Social Work (SHSSW) at UT Austin. We offered the curriculum to all MS2s during their 4-week emergency medicine clerkship, which took place in 53- and 27-bed EDs in urban, safety net academic medical centers in Texas. Participation was voluntary; however, it was incentivized through indications on final clinical evaluations.

Curriculum Activities

This three-part curriculum consisted of a 1-hour didactic session, a formative OSCE, and one SBIRT session delivered to students during their service in the ED. The didactic portion consisted of a presentation (Appendix A) led by an interprofessional panel, consisting of a third- or fourth-year medical student and a behavioral scientist, nurse, or psychologist. The upper-level medical student discussed validated screening tools, including the Alcohol Use Disorder Identification Test (AUDIT; Appendix B) and Drug Abuse Screening Test (DAST) (Appendix C), how to address stigma associated with substance use, and how to provide feedback to patients, including safe drinking level guidelines as defined by the United States Preventative Task Force. The behavioral scientist, nurse, and psychologist are experienced trainers belonging to the Motivational Interviewing Network of Trainers (MINT).^18^ They, in turn, trained the students in MI techniques to employ during SBIRT interviews, such as the stages of change, MI techniques outlined in the OARS Model (open-ended questions, affirmations, reflections, summarization), and the Readiness Ruler for promoting readiness to change talk. Finally, we provided an SBIRT algorithm (Appendix D) to guide our students through a typical interview for use during OSCEs or the delivered SBIRT administration during their ED rotation.

OSCEs took place in simulation centers on the Dell Seton Medical Center (Dell Med) and UT School of Nursing campuses and consisted of three 12-minute interviews with SPs. The SPs were volunteer graduate students from health profession schools at UT Austin who had previously completed advanced coursework in behavioral health and MI. Each SP was given one of three available character descriptions with associated AUDIT/DAST scores (Appendices E and F) and OSCE logistics via email. They were instructed to improvise their responses to students’ SBIRT administration based on their character descriptions and to demonstrate willingness to change. We developed each OSCE case and character description collaboratively between students from Dell Med and faculty from the SHSSW. Cases included SPs of various ages and mock medical conditions that commonly present to the ED. Each case included risky levels of drug or alcohol use. We instructed students to review the SP's AUDIT or DAST score, discuss their risk category, administer a brief intervention utilizing MI techniques, and determine the appropriate next steps based on the SP's readiness to change during each case (Appendix G). During their encounters, we also encouraged students to use informational materials including the SBIRT algorithm (Appendix D), a Substance Use Fact Sheet (Appendix H), and a Brief Intervention Card developed by the Office of Addiction Services and Supports (Appendix I). We monitored students’ performances virtually and our MI coaches scored them according to their use of OARS MI techniques during each OSCE. Virtual observation was made possible via live-streaming audio and video captured in simulated hospital rooms to laptops monitored by our coaches. Following each encounter, 10 minutes of individualized formative feedback was provided by trainers from MINT. We typically ran two simulations at once, each followed by feedback, requiring three to four actors (one for each case), and two MI coaches. A sample schedule is provided in Appendix J.

SBIRT Delivery

After completing the didactic and OSCE portions of the curriculum, we asked students to identify an appropriate patient presenting to the ED at either Dell Med or the Seton Medical Center in Austin, Texas, with whom to administer the SBIRT intervention. Patients who were actively intoxicated, in distress from withdrawing from a substance, combative, hemodynamically unstable, experiencing altered mental status, or unable to talk due to oxygen requirements were ineligible for SBIRT intervention. Student-patient interactions were supervised by ED residents and attending physicians. Upon completion of SBIRT administration with at least one patient, students coordinated with the patient and ED social workers to determine appropriate referral or provision of information regarding available resources according to the patient's insurance status.

We attempted follow-up calls to patients who received the SBIRT intervention during their ED encounter, at 2 weeks and 4 weeks after discharge, if they consented to follow-up contact. We planned to ask patients both open- and closed-ended questions about any behavior changes, whether they followed up with the referred resources, and any barriers they encountered. These follow-up calls were performed by either a member of the project team or the student who had administered the SBIRT. We provided a script to students to facilitate follow-up call conversations and to provide them with appropriate responses to patient queries around medical care or care coordination (Appendix K). These questions were to be noted and emailed to the project team, including an attending physician, to review and address with the patient separately. In addition, patient responses to follow-up calls were to be separately recorded in a Student SBIRT Patient Follow Up Survey (Appendix L). If patients changed their behavior or engaged with the resources to which they were referred, this was considered an additional indicator of student SBIRT administration proficiency.

Patients were free to refuse SBIRT administration from students. Regardless of patient participation in students’ SBIRT delivery, patients received standard of care for their chief presenting symptoms and any additional medical concerns noted during evaluation. The UT Austin Institutional Review Board deemed this activity non-human subjects research.

Learner Assessment

We conducted learner assessments across Levels 1 through 3 of Kirkpatrick's learning evaluation model.^19^ At Level 1, we assessed student attitudes and preparedness in using MI techniques and administering SBIRT, through self-reported pre- and postcurriculum surveys (Appendix M). Level 2 focused on evaluating students’ knowledge acquisition from pre- to postcurriculum, using multiple-choice assessments covering lecture topics (Appendix N). For Level 3, we measured student behavioral changes by assessing score trends across three OSCEs, the completion of a Student Administered SBIRT Form (Appendix O) after the student had administered a simulated SBIRT interview, and patient-reported outcomes via the Post-SBIRT Patient Feedback Form (Appendix P). We used all assessments as curriculum evaluation methods. The scores on all evaluation methods did not impact clerkship grades.

We developed pre- and postcurriculum attitudes and preparedness surveys (Appendix M) in collaboration with faculty members of the Department of Internal Medicine at Dell Med. These surveys, consisting of nine items, assessed students’ attitudes toward substance use disorder and sense of preparedness around MI and SBIRT administration. We scored each item on a 6-point Likert scale (scores ranging from 1 to 6; 1 = strongly disagree, 6 = strongly agree) based on the extent to which the student agreed with each statement. We scored surveys by summing the scores for all items; these summed scores ranged from 9 points to 54 points, with higher scores indicating increased preparedness and more positive attitudes.

We developed the pre- and postcurriculum knowledge assessments (Appendix N) using an iterative approach in collaboration with faculty members at the SHSSW and Dell Med. We designed them to assess students’ knowledge of didactic topics, including defining severity of substance use disorder, stages of change, and MI techniques.

We evaluated the OSCE cases with a score sheet derived from the MINTs’ resident physician training (Appendix Q), which is an adaptation of the Motivational Interviewing Treatment Integrity Coding System 4.^20^ MINT coaches graded students based on their students’ exhibition of MI spirit (e.g., partnership, acceptance, and compassion), empathy during the SP encounter, and the number of times they utilized MI techniques and provided open-ended feedback. We analyzed student performance by comparing the score change from round one to round three. In between sessions, MINT trainers offered formative feedback to improve students’ comfort and skill with interviewing patients.

After a student administered an SBIRT interview with a patient in the ED, they completed the Student Administered SBIRT Form (Appendix O), which documented AUDIT/DAST scores, MI techniques used, referrals to social work, and a narrative summary of the interview. In addition, students referred unit case management staff to administer the Post-SBIRT Patient Feedback Form (Appendix P). We developed this survey in collaboration with Dell Med faculty members. The survey contained three items relating to the students’ endorsement of MI spirit, as defined by MINT during the SBIRT administration from the patient's perspective.^14^ We scored the patient survey responses on a 6-point Likert scale based on greater agreement with each survey item (1 = strongly disagree, 6 = strongly agree). The maximum achievable score on the patient survey was 18. To increase student engagement with SBIRT delivery in the ED, students who completed this portion of the curriculum were able to receive credit on their final clinical evaluation (Appendix R). To that end, given that multiple steps were required to proceed through our curriculum and claim credit per our incentive, we provided students with a comprehensive workflow document (Appendix S).

Statistical Analysis

We used the Wilcoxon signed-rank test as a nonparametric statistical test for comparing pre- and postcurriculum evaluation results, as well as OSCE scores. Given the small sample size throughout each evaluation, we used nonparametric testing for statistical analysis, with the significance level set at p < .05. We generated all analyses using Microsoft Excel.

Results

A total of 56 MS2s participated in the curriculum, with 20 administering formal SBIRTs in the ED.

We excluded from the survey analysis students who did not complete both the precurriculum and postcurriculum evaluations (knowledge assessment and other surveys). Of the 56 students, 41 MS2s completed both the pre- and postcurriculum surveys on attitudes and preparedness. Among these respondents, the median total score of students’ attitudes and preparedness increased by 16% (SD = 9%).

Due to question length and difficulty, we revised the knowledge assessment twice, with the third version leading to more consistent results. Students who completed the first two versions of the knowledge assessment (n = 22) or who did not complete both rounds of the third version (n = 10) were excluded. Of the remaining 24 students, the median precurriculum knowledge assessment score was 67%, while the median postcurriculum knowledge assessment score was 78%. Student's median total knowledge assessment score increased by 11% (SD = 15%).

Of the 56 students who participated in the SBIRT curriculum, 37 (66%) completed the OSCE portion. Students’ median OSCE score increased by 19% (SD = 35%) from the first to the third SP encounter.

In comparing results of the pre- and postcurriculum assessments using the Wilcoxon signed-rank test (Table), we found that there was a statistically significant difference in median scores between the pre- and postcurriculum attitudes and preparedness survey (p < .001) and knowledge assessment (p = .002), and a statistically significant increase in median scores between the first and third round of OSCE scores (p = .03).

Table.: Changes in Learner Assessment Scores Before and After Participation in the SBIRT and MI Curriculum

Of the 20 students who administered an SBIRT interview to a patient, 12 (60%) provided a narrative of their SBIRT interview, including MI techniques used. Techniques used by students included assessing readiness/Readiness Ruler, open-ended questions, affirmations, elicit-provide-elicit, building rapport, reflective listening, and exploring ambivalence. Twenty-four percent of students utilized assessing readiness and/or the Readiness Ruler, 17% utilized reflective listening, open-ended questions, and building rapport, 11% utilized affirmations, 7% of students utilized exploring ambivalence and elicit-provide-elicit techniques (Figure).

Motivational interviewing (MI) techniques used during the screening, brief intervention, and referral to treatment (SBIRT) curriculum. Twenty students administered SBIRT to a patient, and 12 (60%) provided a narrative of their SBIRT interview, including MI techniques used.

Of the 20 patients who underwent SBIRT administration, six requested follow-up phone calls. Three of those six provided a phone number with which to contact them, but we could not reach them via phone at their preferred follow-up time.

Discussion

Through the development and implementation of this novel integrated medical student SBIRT curriculum, we have demonstrated an increase in knowledge and proficiency in SBIRT administration and MI techniques. Collectively, our attitudes and preparedness survey, knowledge assessment, and OSCE data indicate that formal MI and SBIRT training in clerkship medical education of MS2s can significantly impact the students’ proficiency and comfort with SBIRT and MI. There was attrition demonstrated in students’ pursuit of SBIRT opportunities in the ED, and therefore our ability to claim student proficiency based on their narrative experiences and patient outcome data is limited. It is questionable whether the ED is the best milieu for students to leverage their SBIRT training, especially given the preexisting cognitive and logistical demands of emergency medicine for MS2s and our curriculum's designation as optional. The attitudes and preparedness survey results, however, suggest that exposing medical students to MI and SBIRT techniques early in their careers may improve their comfort level with addressing substance use with patients. When equipped with SBIRT, and after gaining proficiency with the SBIRT simulation, learners are more likely to engage in these sensitive discussions than before their training. This could have positive implications for patient outcomes.

Challenges and Limitations

Educators considering replication of our curriculum should consider the following challenges, including their associated iterations and suggestions for overcoming them. Challenges included revising knowledge assessments, addressing large standard deviations in OSCE score changes, encouraging SBIRT administration in the ED, incentivizing general participation in a noncompulsory curriculum, and acquiring patient follow-up data.

Our pre- and postcurriculum knowledge assessments underwent two revisions due to question difficulty and length. The original assessment questions required students to recall information they would not need to memorize for SBIRT administration. The revised assessment posed questions with responses that reflected students’ practical understanding. These revisions halved the student assessment scores used in our data analysis, which limits the results’ statistical power to claim definitive proficiency. We are reassured that student declarative understanding is corroborated by OSCE score improvements over time, which is supported by a larger sample size. Replications of this curriculum should evaluate whether there is a correlational relationship between OSCE and assessment scores.

The OSCE score changes between first and third SP encounters exhibited large standard deviations. Possible explanations include unclear student case instructions, SP acting variability, and interrater reliability for MI spirit and empathy scores—which did not have a defined rubric—and differences in rater ability to correctly tally MI technique “counts.” Solutions include eliciting feedback from low-performing students on how to improve our instructions, standardizing actor performances through more robust training or protocols, creating a rubric for scoring ambiguous items during the OSCE, and recording simulations to improve accuracy in tallying “counts.”

Although not formally measured, general participation in the curriculum was limited by the curriculum's optional nature, requiring different strategies to encourage student engagement. An implied solution for replicating similar curricula would be to make it compulsory or available via elective opportunities. However, we adapted a few strategies to increase participation. We informed ED attending physicians of our pilot program to increase their awareness of it to facilitate student participation. We incentivized students to perform SBIRT in the ED by offering to boost their clerkship evaluations according to a standardized rubric (Appendix R). Despite these efforts, a challenge that persisted was that students cited difficulty identifying patients who would score high enough on AUDIT/DAST to warrant brief interventions but who were also not so intoxicated that their participation was excluded. To help with this patient identification barrier, we recruited ED social workers to refer appropriate patients to students, but this effort was challenged by ED social worker bandwidth.

It was difficult to obtain patient follow-up data from the patient satisfaction surveys and the follow-up phone calls. Patient satisfaction survey collection suffered due to coordination issues between students and ED social workers, and patients not answering follow-up phone calls. Consequently, this data did not yield meaningful results and was excluded from our analysis. Further, follow-up calls could have been confounded by bias when completed by administering students if patients decided to overreport SBIRT impact to protect students’ feelings.

Given our challenges related to student participation in the ED and patient survey data collection coordination, this curriculum would be better administered as an elective opportunity. While this would limit the number of participants, student self-selection would reduce attrition. A focused elective in which students respond to inpatient or ED SBIRT consults would lessen cognitive load. Coordination of patient outcome data could be improved by reassigning data collection to clinical supervisors rather than social workers with competing duties.

This curriculum relies heavily on a strong partnership with the SHSSW, the Health Behavior Research and Training Institute, and its trainers from the MINT program. Further, this project's funding to pay MINT- certified MI coaches during our OSCE was contingent on trainees’ use of SBIRT in inpatient settings. If grant funding stipulations for this work were more flexible, this curriculum could be replicated by other institutions anywhere.

Further Directions

Moving forward, given the experienced challenges, our aim is to integrate this vetted curriculum into a preexisting addiction medicine elective concurrently developed by internal medicine residents at Dell Med. There is expressed interest in incorporating our curriculum into this elective, which could be extended to MS2s and MS4s who have elective time available in their respective years. Formal integration would achieve access to allocation of instructional funds from the Department of Internal Medicine to supplant the grant funding the curriculum relies on.

Appendices

• Medical Student MI-SBIRT Curriculum.pptx
• Alcohol Use Disorder Identification Test.docx
• Drug Abuse Screening Test (DAST-10).docx
• SBIRT Algorithm.docx
• SP Case Descriptions.docx
• SP Case.docx
• Student OSCE Instructions.docx
• Substance Use Facts Sheet.docx
• SBIRT Brief Intervention Card.docx
• Sample OSCE Schedule.xlsx
• Patient Follow-Up Guide.docx
• Student SBIRT Patient Follow-Up Survey.docx
• MI-SBIRT Attitudes and Preparedness Survey.docx
• Pre- and Postcurriculum Assessment.docx
• Student-Administered SBIRT Form.docx
• Post-SBIRT Patient Feedback Form.docx
• OSCE Score Sheet.docx
• Exceeds Criteria.docx
• Student Workflow and Protocol.docx

All appendices are peer reviewed as integral parts of the Original Publication.