Evaluating the Role of Objective Structured Clinical Examination as a Summative Assessment Tool in Undergraduate and Postgraduate Psychiatry Residents

TL;DR

This study evaluates how students and examiners perceive the use of OSCE in psychiatry training, finding generally positive views and suggesting ways to improve the process.

Contribution

The study provides insights into the effectiveness and perception of OSCE in psychiatry, a specialty where its use is limited.

Findings

Most students and examiners had positive perceptions of OSCE's validity and reliability.

Thematic analysis highlighted improved confidence in teaching and feedback among faculty.

Improved preparation and orientation could reduce anxiety related to OSCE.

Abstract

Background: The Objective Structured Clinical Examination (OSCE) is the gold standard and universal format to assess medical students' clinical competence in a comprehensive, reliable, and valid manner. OSCE is gaining global popularity for assessing medical students in various specialties. Our country uses it in multiple disciplines, but its application in psychiatry remains limited. OSCE is a resource-demanding assessment method that can face numerous challenges. A comprehensive assessment of perceptions regarding OSCE can help identify areas that need improvement. Therefore, this study was conducted to assess the perceptions of students and examiners toward OSCE in psychiatry.

Aims and objectives: This study aims to evaluate the role of Objective Structured Clinical Examination as a summative assessment tool in assessing competency in undergraduate (as part of their ward leaving examinations in psychiatry) and postgraduate psychiatry residents (as part of their six-monthly assessments) and analyze the perceptions of students as well as of the faculty members regarding OSCE.

Methods: Six MD students and 49 MBBS students took the OSCE as part of their ward-level exams in psychiatry. In the presence of four faculty members of the psychiatry department, students completed their six-month summative exams. The OSCE was held at separate times for UG and PG students. UG and PG students utilized different stations (PG students had a harder level). A 10-item questionnaire was given to examiners and students at the end of the OSCE to get their opinions regarding the OSCE. Furthermore, data gathered from the faculty through an open-ended questionnaire was compiled and displayed thematically. Since the Likert scale survey generated ordinal data, the statistical analysis was conducted using the median, interquartile range (IQR), and chi-square test. The chi-square test was used to compare the variables. A P-value of less than 0.05 was deemed statistically noteworthy.

Results: Four faculty members and fifty-five students in all answered the questionnaire. Regarding the OSCE's characteristics, validity, reliability, and transparency, the majority of students expressed positive opinions. In a similar vein, most examiners had positive perceptions pertaining to OSCE's administration, structure, and procedures. Certain areas, such as "improved confidence in teaching clinical skills" and "improved confidence in giving students structured feedback," were also emphasized by thematic analysis of faculty members.

Conclusion: In general, both students and examiners had extremely favorable perceptions of and embraced the OSCE. Improved faculty orientation and student preparation for the OSCE may help allay anxiety and overcome hesitation related to the exam.

Figures