Evaluating the Clinical Learning Environment for Competency-based Postgraduate Education in a Low-Middle income Country: Trainee Perceptions using PHEEM Inventory

TL;DR

This study evaluates trainees' perceptions of their clinical learning environment in a low-middle income country to support competency-based postgraduate medical education.

Contribution

The study provides insights into the strengths and weaknesses of the clinical learning environment for competency-based education in a specific regional context.

Findings

Trainees generally perceive the clinical learning environment favorably, with room for improvement.

Residency programs and years show significant variation in perceived quality of the clinical learning environment.

The PHEEM inventory effectively identifies areas needing enhancement for competency-based education.

Abstract

The clinical learning environment (CLE) is an important element of competency-based postgraduate medical education (CBME). Trainee perceptions of their CLE serve as a quality indicator of the teaching and learning taking place at the workplace. This study aimed to investigate the trainees’ perceptions regarding their CLE and identify strengths and weaknesses to support CBME.

A universal sampling was employed for this cross-sectional survey. The electronic version of the Postgraduate Hospital Educational Environment Measure (PHEEM) inventory was sent via email to all the trainees who were enrolled in all 35 residency programs at our university and consented to participate in the study.

A total of 347 (69.4%) residents responded; of them, 65.7% were females. The overall mean score was 107 (66.8%), suggesting a generally favorable opinion of the workplace, with room for improvement. Mean scores for the subscales of Autonomy, Teaching, and Social Support were 33.42±7.24 (more positive), 42±8.9 (moving in the right direction), and 27.9±6.2 (more pros than cons), respectively. There was no difference in the CLE perceptions based on gender. Overall and subscale scores differed significantly across residency programs, with the highest in Radiology (122.3±13.5) and the lowest in Surgical disciplines (95.47±19.0); and year of residency, with the highest in the first year (111.3±17.8) and the lowest in the final year (81.5±34.3).

Evaluating the EE offers valuable insights into enhancing training quality by identifying both strengths and weaknesses, and prioritizing areas in any planned enhancements. The EE for the postgraduate training appears to be prepared for implementing CBME, with certain areas warranting improvement.

Tables

Table 1. Overall PHEEM and subscales interpretation key and scores

Domain (# of items)	Max score	Mean±SD (%)	Range	Interpretation key
Overall (40 items)	160	107±21.4 (66.8%)	32 – 160	<40: Very poor
				41–80: Plenty of problems
				81–120: More +ve than -ve but room for improvement
				>120: Excellent
Perceptions of Autonomy (14 items)	56	33±7.24 (58.9%)	9 – 52	0–14: Very poor
				15–28: Negative view of role
				29–42: More positive perception of one’s job
				43–56: Excellent perception of one’s job
Perceptions of Teaching (15 items)	60	42±8.9 (70%)	11 – 60	0–15: Very poor
				16–30: Need some re-training
				31–45: Moving in the right direction
				46–60: Model teachers
Perceptions of social support (11 items)	44	27±6.2 (61.3%)	7 – 44	0–11: Non-existent
				12–22: Not a pleasant place
				23–33: More pros than cons
				34–44: Good supportive environment

Table 2. Overall and subscale scores by residency program

Discipline	N (%)	Overall (160)	Autonomy (56)	Teaching (60)	Social Support (44)
Radiology	12 (3%)	122.3±13.6	39.2±4.7	48.6±5.2	31.5±5.1
Pathology	27 (8%)	119.1±16.8	38.4±4.6	47.3±6.7	30.3±5.8
Dentistry	14 (4%)	115.3±25.0	39.0±9.1	45.1±9.0	28.4±7.4
Family Medicine	29 (8%)	112.4±19.1	35.5±6.4	44.6±8.8	29.2±5.4
Pediatrics	24 (7%)	106.5±18.1	34.3±7.4	38.7±9.7	26.3±4.9
Medicine & allied	109 (31%)	106.4±22.4	32.8±7.5	42.0±9.2	28.6±6.4
Emergency Medicine	26 (7%)	104.5±23.3	33.8±35.5	41.7±10.0	25.7±7.1
Anesthesia	43 (12%)	99.8±21.5	32.0±6.5	39.0±9.4	26.3±6.9
Obstetrics and Gynaecology	17 (5%)	99.0±14.6	30.3±5.0	39.4±6.8	26.9±4.3
Surgery & allied	46 (13%)	95.5±19.1	28.8±6.4	38.1±8.4	26.1±5.3
P		<0.001	<0.001	<0.001	<0.021

Table 3. Overall and subscale scores by residency year

Year of Residency	N (%)	Overall	Autonomy	Teaching	Social Support
1	103 (30%)	111.3±17.8	34.8±6.1	43.8±7.7	29.7±5.0
2	79 (23%)	100.5±20.3	31.6±7.1	39.6±8.9	26.6±5.5
3	63 (18%)	105.3±20.2	33.8±7.1	42.0±8.4	26.9±5.8
4	54 (16%)	108.2±20.8	33.9±7.0	42.9±8.7	28.3±6.5
5	42 (12%)	106.3±27.5	33.5±8.9	42.1±10.4	27.9±8.5
6	6 (2%)	81.5±34.3	25.3±10.8	31.3±13.8	23.2±10.1
P value	-	0.001	0.005	0.001	0.004

Table 4. The items with the highest and lowest scores

Areas perceived as Strengths (score ≥3)
4. I had an informative orientation program.	POA	3.0
13. There is no gender discrimination in this post.	PSS	3.0
29. I feel part of a team working here.	POA	3.01
5. I have the appropriate level of responsibility in this post.	POA	3.02
6. I have good clinical supervision at all times.	POT	3.03
31. My clinical faculty are accessible.	POT	3.04
10. My clinical faculty have good communication skills.	POT	3.04
18. I have the opportunity to provide continuity of care.	POA	3.1
12. I have good collaboration with other trainees in my residency year.	POT	3.2
24. I feel physically safe within the hospital environment.	PSS	3.44
Areas perceived as weaknesses (score ≤ 2)
32. My workload in this job is fine.	POA	1.76
17. My working hours conform to PGME (80hrs/week) policy.	POA	1.93
11. I am paged inappropriately.	POA	1.96
19. I have suitable access to career advisor.	PSS	2.0
36. I get a lot of enjoyment out of my present job.	PSS	2.02
38. There are good counselling opportunities for junior doctors who fail to complete their training satisfactorily.	PSS	2.04

Full text

Introduction

The clinical learning environment (CLE) is a vital element in postgraduate medical education. It refers to the context in which trainees learn to apply theoretical knowledge and hone their skills while they are involved in patient care and service provision. In other words, it is “the overlapping space between the educational context (the syllabi, curricula, and goals that define learning outcomes, methods of learning, and assessment practices) and the clinical work environment, this also includes institutional cultures & processes, physical and virtual spaces and social interactions that influence how learners experience and perceive learning ( 1

• 3 ).

The influence of the CLE on the quality of training programs, and trainees’ satisfaction, clinical performance, and academic achievement has been well-documented in the literature. The quality of the learning environment that provides the context for training also predicts the quality of care, prescribing patterns, and use of healthcare resources by graduates, years after graduation ( 4 ). An educational environment that is perceived as being conducive not only motivates learning but also ensures positive program outcomes and improved patient care ( 1 ). In contrast, a negative learning environment not only hinders professional growth of the trainees but also disrupts interpersonal relationship and team work that are essential for delivering optimal healthcare ( 2 ). Its significance has been further emphasized within the framework of CBME ( 3 ). It has been identified as the strongest predictor of preparedness for practice, followed by attention to competencies ( 5 ). A good clinical learning environment provides more opportunities for experiential learning and ensures that teaching and learning are relevant to patient care. In such contexts, there is active participation by learners who demonstrate professional thinking and behavior, and more support for learners to reflect on and learn from these experiences that leads to the development of competent clinicians ( 6 ). Programs considered as supportive of their trainees are perceived as having an optimal learning environment, while negative perceptions of the CLE have been associated with trainee burnout and suboptimal learning outcomes ( 3 ).

The Aga Khan University (AKU) is one of the prestigious institutes of Pakistan that offers over 30 residency programs, all of which are recognised by the College of Physicians and Surgeons, Pakistan (CPSP). In keeping with the AKU’s mission of developing competent physicians to meet healthcare needs, the Office of Postgraduate Medical Education (PGME) adopted the competency-based educational framework for its residency programs. Several initiatives were taken at the institutional level over the last few years to improve the clinical learning environment and facilitate the implementation of CBME. The potential interventions included ascertaining competencies for trainees at various levels, implementation of institutional supervision policy, continuous cognitive and workplace-based assessments for trainees, work hour regulations, and faculty development in teaching and assessing trainees. Being in a low-middle income country, there was a need to measure the influence of these interventions on the CLE and the resultant trainee perceptions at our institution.

This study aimed to measure the trainees’ perceptions regarding their learning environment at the clinical workplace with the objective to identify the strengths that needed to be reinforced, and measures to be taken to improve areas of weakness to support competency-based residency education.

Methods

Study Design and Participants

This cross-sectional study was conducted between 2022 and 2023 at AKU to measure the trainees’ perceptions of their clinical learning environment. A universal sampling was employed, inviting all the trainees enrolled across all 34 residency programs at AKU to participate in the study. The duration of various residency programs at AKU ranges from 4-6 years as specified by the national accrediting body, the College of Physicians and Surgeons of Pakistan (CPSP). Trainees enrolled in any residency program at AKU, for at least 3 months at the time of data collection, were included in the study. This time duration ensured that participants had adequate exposure to the CLE at AKU to form meaningful perceptions. Trainees who had spent less than three months at the time of data collection, or who had submitted incomplete forms were excluded from the study.

Data collection instrument

The Postgraduate Hospital Educational Environment Measure (PHEEM) inventory is a 40-item self-reported inventory developed to measure the postgraduate clinical learning and teaching environment of junior doctors ( 7 ). It measures learners’ perceptions during the hospital-based training period in three domains, i.e. perception of autonomy (POA) [item # 1, 4, 5, 8, 9, 11, 14, 17, 18, 29, 30, 32, 34, and 40], perception of teaching quality (POT) [item # 2, 3, 6, 10, 12, 15, 21, 22, 23, 27, 28, 31, 33, 37, and 39], and perception of social support (PSS) item no. 7, 13, 16, 19, 20, 24, 25, 26, 35, 36 and 38] in the clinical educational environment, and are known as subscales of the instrument.

PHEEM is a valid and reliable instrument and has been one of the mostly used tools for postgraduate programs globally, to help identify areas of strength and prioritize areas for planning improvement in domains of autonomy, teaching, and social support ( 8 , 9 ). Its validity has also been demonstrated previously in the Pakistani context, with local studies reporting high internal consistency (Cronbach’s alpha≥ 0.8) ( 10 , 11 ).

Two minor modifications were made in the instrument to ensure local relevance; in item 7, ‘racial’ was replaced with ‘ethnic’ because all the trainees at AKU are of the same race but from different ethnic origins. Additionally, ACGME was replaced by PGME in [item #17. The instrument was piloted before administration to ensure comprehension, relevance to the local context, and accessibility using online modality, and was found to be appropriate for data collection.

Data Collection

The study complied with the ethical standards established by the Declaration of Helsinki. All eligible participants were sent an invitation email explaining the study’s purpose, procedure, and voluntary nature of participation. Participants were assured of confidentiality, anonymous data handling, and the right to withdraw at any stage, and requested to provide written informed consent for participation. Those who consented were provided with a link to the Postgraduate Hospital Educational Environment Measure (PHEEM) inventory as a Google form. Basic demographic data of trainees, such as residency program, gender, and year of residency, were added in the survey. Reminders were sent after every few days for 3 months to ensure maximum participation. Only forms that were complete were used for analysis, while the incomplete forms were excluded.

Data Analysis

Each of the 40 items of PHEEM questionnaire are scored on a 5-point Likert scale (0-4) where 0=strongly disagree and 4=strongly agree. The 40 items yield a total maximum score of 160, which is indicative of the ideal hospital-based educational environment. Four out of 40 items [7, 8, 11 and 13] are negatively phrased statements and are scored in reverse order, i.e. 0 for strongly agree, 1 for agree, 2 for unsure, 3 for disagree, and 4 for strongly disagree. Items with a mean score greater than 3 mainly represent strong areas, while those with a mean score of less than or equal to 2 are indicative of problem areas, thus requiring immediate review and remediation. Items with a mean score between 2 and 3 reflect areas that are neither strengths nor weaknesses, but areas that could be enhanced. The Practical Guide described by the authors ( 7 ) for interpreting the overall and subscale scores, and the number of items in each subscale are shown in Table 1.

Descriptive analysis included mean, standard deviation (SD), and percentages for quantitative variables, while frequencies for qualitative variables were calculated to summarize the data. Reliability of the instrument was measured using Cronbach’s alpha. Kruskal-Wallis one-way ANOVA with post hoc Dunn test for differences between groups was used to compare the differences between residency programs. SPSS statistical software (Version 20, IBM Corporation, Armonk, NY) was used at the significance level of less than 0.05 (P<0.05) with a confidence interval of 95%.

Ethical Considerations

Approval was obtained from the institutional Ethics review committee (ERC#: 2021-5536-19837) before initiating the data collection. The study was conducted in compliance with the Declaration of Helsinki, establishing ethical standards. Consent was obtained from the participants, and the survey forms were sent only to those who consented. The identity of the participants was anonymized, and data were kept confidential by restricting access to the PI only.

Results

A total of 347 out of 500 trainees completed the survey, giving a response rate of 69.4%.  Of these, only 119 (34%) were males, with females making up 65.7% of the sample. Of the total responses, the highest response was received from the first-year trainees (30%), followed by the second year (23%), while only 6 (2%) trainees in the sixth year of residency responded to the survey. Most of the respondents were from the Medicine and allied programs (31%), followed by Surgery and allied residency programs (13%), while Radiology and Pathology trainees were the least in numbers, being 3% and 4% of the total respondents, respectively. Reliability, measured as Cronbach’s alpha, was 0.95, and the exclusion of individual questions did not produce significant changes in the score, reflecting high internal consistency.

The mean total PHEEM score was 107 (66.8%) out of a maximum score of 160, suggesting that although trainees have a generally favourable opinion of their workplace, there is still room for improvement. The overall PHEEM and subscale (mean) scores are shown in Table 1, and the corresponding interpretation marked bold.

Post-hoc analysis using Dunn's test with Bonferroni correction revealed significant differences between all three domains (p<0.05). The teaching domain scored significantly higher compared to both Social Support (p<0.001) and Autonomy (p<0.001). Social Support scores were also significantly higher than Autonomy scores (p=0.0146).

There was no significant difference in the overall and subscale mean scores based on gender. However, statistically significant differences were observed in the overall satisfaction and subscale scores across residency programs, as determined by the Kruskal-Wallis one-way ANOVA (p<0.001 for overall, autonomy, and teaching; p=0.021 for social support), as shown in Table 2. Post-hoc pairwise comparisons between programs showed significant variations across all domains (Autonomy: H=57.18, p<0.001; Teaching: H=32.55, p=0.0002; Social Support: H=24.68, p=0.0034), with Radiology consistently showing the highest scores and Surgery and allied programs the lowest across all domains.

The Kruskal-Walli’s test results showed differences across PGY levels for all domains with most trainees having better perceptions about their clinical learning environment compared to the senior trainees (p=.001) (Table 3). There was a noticeable trend toward increasing Autonomy scores in higher PGY levels, particularly in surgical specialties. Teaching scores remained relatively stable across PGY levels, suggesting consistency in educational delivery throughout training years.

The authors of PHEEM recommend that any item with a mean of 2 or less should be examined in detail, as it may indicate a problem area, while items with a mean score of 3.5 or over indicate real positive points. Table 4 provides a summary of the items with a mean total score below 2 and above 3 points in the study. The highest scoring items were items # 12 and 24, while the lowest scoring items were items # 32, 17, and 11. Table 4 shows the highest and the lowest scoring items.

Discussion

Clinical learning environment is a multifaceted and dynamic construct that is influenced by individual, interpersonal, and institutional influences. It plays a pivotal role in shaping the postgraduate trainees’ experiences and perceptions, and their development as physicians. Understanding the trainees’ perspectives regarding their CLE can provide insightful suggestions for enhancing the learning environment and fostering a supportive workplace.

The mean overall PHEEM scores [107] of all the trainees in all the residency programs in our study had ‘more positive perception’ about their learning environment at the workplace, which is similar to other studies reported from the Asian countries including the Emergency Medicine and Urology residency programs in Saudi Arabia (109.9 and 98.2, respectively), Paediatric Specialist Training Program in Indonesia (108.10±17.03), and Internal Medicine program in Singapore (112.23±16.7), However, this is better than what has been reported from other institutions in Pakistan where the overall mean PHEEM scores ranged from 63.06 (±16.77) in a pediatric surgery residency program ( 12 ), 63.68 (±29.60) in the Obstetrics and Gynecology residency program in Lahore ( 13 ) and 64.1 (±29.7) in Radiology residency program in Islamabad ( 14 ) to 97.29 and 104.7 in General Surgery and Psychiatry residency programs, respectively, in Hyderabad ( 15 ). One exception includes a PHEEM score of 107.29±55.73 in the Obstetrics and Gynaecology residency program in Rawalpindi ( 16 ).

We did not find any difference based on gender. This is similar to most studies ( 14 ), though few studies had also reported a difference in the perceptions of learning environment based on gender where either males or females perceived the learning environment better as compared to the other gender ( 17 , 18 ). A study from Pakistan also reported a significantly higher mean PHEEM score among females ( 12 ). Lack of influence of gender on the PHEEM scores in our study indicates a learning and work environment that is perceived to be equally safe, respectful, and supportive for all its male and female trainees. However, this could also be because more than 50% of the respondents were females.

Our study showed that junior trainees perceived a significantly more favorable learning environment, as compared to their senior peers. A noticeable trend suggests that perceptions were best in the first year of residency training; then, they declined in the second year, improving again later during training, but never reaching that level in the first year. This is similar to other studies ( 8 , 18 ). The decline in the second year in our study could be because of the national accrediting body, CPSP requirements, such as submitting a thesis proposal, and taking their Intermediate module exam. However, there may be other factors that need to be explored, such as an increase in workload and less time for their educational commitments when they are preparing for exams.

Perceptions of Teaching quality (POT) scored the highest (42±8.9; 70%), showing trainees were satisfied with their teaching and learning, and supervision at the clinical workplace. This may be attributed to the introduction of workplace-based assessments that provide opportunities for supervision and feedback from their supervisors, the implementation of supervision policy, and the identification of expected level of competency for independent working in the hospital. This is also evident from the higher-scoring items shown in Table 4 related to POT.

Social support was given a little lower score than POT in our study (27±6.2; 61.3%); however, it is higher than that reported from other studies ( 18

• 20 ). Social support plays a vital role in creating a positive clinical learning environment. It reduces stress, prevents burnout, enhances emotional well-being, promotes knowledge sharing, improves teamwork and communication, and boosts motivation and resilience. This supportive system creates an environment conducive to learning and professional development. Notably, like other areas of CLE, junior trainees had the highest perceptions of the social support available to them as compared to the senior trainees, which calls for attention.

The autonomy subscale (POA) received the lowest scores, possibly due to the rigorous institutional supervision policy implemented across all training sites and levels to uphold patient safety and mitigate potential legal actions against the hospital. However, the items corresponding to the ‘role as a team member’, and ‘level of responsibility’ were among the highly rated items. CBME identifies learner autonomy within the CLE as the fundamental component in the development of independent physicians that builds confidence and trust and promotes their engagement and sense of responsibility for patient care ( 21 ). Studies suggest an intricate relationship between supervision and autonomy, with some studies reporting that excessive supervision can limit the residents' autonomy by hindering their capacity for making independent decisions in managing uncertainty ( 22 , 23 ), while others indicate the unwarranted pressure felt by the residents when expected to operate independently and be self-reliant ( 22 ). It is recommended that the patient care should be balanced with training, and tailor the extent of supervision and autonomy based on residents' competency levels in diverse contexts.

Excessive workload was identified as one of the priority areas to improve the learning environment in our study. It can not only undermine the dedicated education time, but is inversely related to burnout and depression among trainees ( 24 , 25 ). Studies have emphasized the importance of dedicated time for education for postgraduate trainees ( 26 ).

Notably, the perceptions of autonomy and workload, and social support were worst among trainees of surgical disciplines, while perceptions were best among trainees of disciplines like Radiology and Pathology. While this may be secondary to the nature of the work in these specialties ( 27 ), it demands residency program directors and leadership to review the training programs and tailor them where necessary. Another possible reason for this difference could be the lesser number of participants from Radiology and Pathology as compared to other programs.

The clinical learning environment is integral to the success of CBME for postgraduates. It provides the context for the development and assessment of competencies, offers opportunities for hands-on learning, and supports the principles of self-directed learning and continuous improvement. A positive and supportive clinical environment enhances the quality of medical education and prepares the postgraduates to become competent and proficient healthcare professionals. The collective perceptions regarding the CLE, particularly in the context of teaching, may be attributed to the initiatives undertaken by the PGME to implement CBME at AKU. The findings of our study provide valuable insights into the effectiveness of the CBME implementation; it highlights the importance of a clinical learning environment that supports learning, as well as emphasizes considerations for context including year of residency training and discipline when introducing new curricula or educational practices.

Limitation

It's important to acknowledge the limitations of the survey study. First, because the survey relied on self-reported data, there is potential for recall and response bias. Second, the survey was conducted in a specific setting, so its findings may not be generalizable to other medical institutions in Pakistan.

Conclusion

The CLE is a crucial component of postgraduate medical education that provides the context for learning during patient care, maintaining a service education balance. Evaluating the educational environment offers valuable insights into enhancing training quality by identifying both the strengths and weaknesses of a training program and prioritizing areas for improvement in any planned enhancements.