Standardized training assessment of residents in obstetrics and gynecology on entrustable professional activities

TL;DR

This study evaluates the clinical competence of obstetrics and gynecology residents using entrustable professional activities (EPAs) and identifies areas needing improvement.

Contribution

The study introduces a standardized EPA assessment method for OB & GYN residents in China and highlights discrepancies between self and director assessments.

Findings

Residents' EPA scores increased with training years, but PGY1 and PGY2 showed consistent self and director assessments.

Significant differences were found in self-assessment for tasks like admitting patients and performing health education.

Permanent staff scored highest in EPA evaluations, suggesting experience impacts clinical competence.

Abstract

Entrustable professional activity (EPA) presents trainees’ tasks or responsibilities which are entrusted to the unsupervised execution once they have attained sufficient specific competence. EPAs evaluation emphasize the integrity of medical behavior, directly reflecting the clinical ability of trainees and reflect the trust of trainers. Compared with other evalution method, EPAs evaluation is more close to clinical practice and can better reflect clinical competence. Our study conducted a multi-center study in Qilu Hospital of Shandong University to evaluate EPAs among residents. This study was to evaluate EPAs of residents and identify problems in the residency training in Shandong Province, so as to put forward constructive suggestions in futrue.

This is a cross-sectional study. An assessment survey comprised of 15 EPAs was invented and carried out in Department of Obstetrics and Gynecology, Qilu hospital of Shandong Province. Each resident conducted self-assessment and director-assessment through by internet questionnaire of Wenjuanxing.

In our study, 132 residents and 45 directors were totally enrolled. Both self- and director- assessment had an increased scores trend year by year in each EPAs. However, for PGY1 and PGY2 residents, self-assessment and director-assessment results were relatively consistent. Compared with director-assessment, significant difference was found in self-assessment on admit a patient, select and interpret auxiliary examinations, compose medical documents, perform basic operation and perform health education. Among the four different position of residents (social training residents, entrusted training residents, professional master and permanent staff), permanent staff has the highest EPA scores in this EPAs-evaluation study. In this study, we found significant difference between self-assessment and director-assessment in PGY2 and PGY3 (p < 0.05).

EPAs are practically useful in assessment of OB &GYN residents. EPA4, EPA7, EPA1 and EPA15 need to be improved in future training. Individualized feedbacks should be provided based on the difference between self-assessment and director- assessment.

The online version contains supplementary material available at 10.1186/s12909-025-06973-7.

Figures

Comparison of Director-assessment in EPAs of different PGYs between self-assessment and director-assessment. A Comparison of Director-assessment in EPAs of PGY1 between self-assessment and director-assessment. B Comparison of Director-assessment in EPAs of PGY2 between self-assessment and director-assessment. C Comparison of Director-assessment in EPAs of PGY3 between self-assessment and director-assessment. D Comparison of Director-assessment in EPAs of PGY4 between self-assessment and director-assessment. * *p* < 0.05; *p* < 0.01, p* < 0.001Table 5Generalized estimated equation analysis of d

Director-assessment of different position. A Director-assessment of EPAs in permanent staff of different PGYs. B Director-assessment of EPAs in professional master of different PGYs. C Director-assessment of EPAs in entrusted training residents of different PGYs. D Director-assessment of EPAs in social training residents of different PGYs

Director-assessment of different seniority (different postgraduate years). A Self-assessment of different PGYs. B Statistic of different PGYs in self-assessment productive history of patients. It show the gestation, abortion, and partus maturus. C Director-assessment of different PGYs. D Statistic of different PGYs in director-assessment. * *p* < 0.05; *p* < 0.01, p* < 0.001

Full text

Introduction

In China, standardized training for resident doctors is an important process to be qualified doctors [1, 2]. Due to the particularity of obstetrics and gynecology diseases, obstetrics and gynecology residential training should pay a lot of attention to the cultivation of the comprehensive ability of residents. Quantification the clinical ability requirements according to the growth rules of residents, and formation of a training and assessment mode with strong operability and disciplinary characteristics are extremely important to obstetrics and gynecology education research, which is help to better evaluated the ability of residents, consequently, also ensuring patient safety [3, 4].

Entrustable Professional Activities (EPAs) was first proposed by Professor Olle Ten Cate in 2005 [5]. Professional behavior, in particular, refers to the “key” behavior in clinical work. EPAs evaluation is based on the clinical workplace, emphasizing the integrity of medical behavior and the observability in the work environment, which can directly reflect the clinical ability of doctors and reflect the trust of superior doctors [5–7]. Compared with the evaluation method of competency evaluation, EPAs is more close to clinical practice and can better reflect clinical competence [8–10]. Compared with competency model evaluation represented by milestone system, EPAs is easier to be understood by residents and clinical teachers, with simple evaluation criteria and more suitable for evaluation in clinical environment [6, 11]. Since 2005, various training institutions in United States, the Netherlands and Canada have conducted a large number of studies on EPA in various stages of medical education [8, 9, 12–14].

In 2020, based on foreign research background, the first Chinese EPA items for residents was proposed by 38 clinical teaching experts from 11 teaching hospitals of Peking University Health Science Center [10]. Used this establishment system, wchich including 15 EPAs, we conducted a evaluation study based on EPAs among residents. In this study, we purposed to preliminarily explore EPAs-evaluation effect, and to identify the existing problems in the clinical training of resident training in Shandong Province, so as to put forward constructive suggestions. EPAs can be used for formative evaluation and final evaluation [6, 7, 10]. Clinical teachers and teaching management departments can grasp the training situation in real time, conduct dynamic regulation and personalized training, ensure the quality of training, and finally achieve the standardized and homogenized training goals [10, 15].

Qilu hospital of Shandong University has totally 4,360 beds, more than 210,000 discharge visits, and 170,000 operations per year. In department of Gynecology and Obstetrics, there are 357 beds, including gynecologic oncology, minimally invasive gynecology, gynecologic urology, gynecologic endocrinology, perinatal medicine, family planning, reproductive medicine seven specialties. The annual outpatients of obstetrics and gynecology is more than 270,000, including a considerable number of cases are critical or difficult cases. There was a total number of more than 18,000 inpatients and 3400 deliveries in 2023.

Methods

In our research, we took all the residents in Department of Obstetrics and Gynecology into our study. Totally, 147 residents were involved, including 100 residents who were in the rotational training, as well as 47 graduated residents who just finished standardized training. We conducted a cross-sectional study with these 147 residents, and finally 132 residents finished both self-assessment and director-assessment by internet questionnaire of Wenjuanxing, a network questionnaire website. There are 15 residents did not have both self-assessment and director-assessment because some reasons. Among these 15 residents, 7 graduated residents did not reply the self-assessment, 5 residents had only 1 piece of director-assessment and 3 residents were rotating in other departments (Ultrasonic Department and Anesthesiology Department).

Setting

According to the national guidelines for standardized residency training, every resident of Gynecology and Obstetrics Department is supposed for different postgraduate years (PGYs), including PGY1 (who has been residency training lease than 3 months), PGY2 (who has been residency training more than 1 year but less than 15 months), PGY3 (who has been residency training more than 2 year but less than 27 months), PGY4 (who has just finished the 3-year training phase). We Create wechat groups for each PGY, then collected the self-assessment questionnaires from residents of each PGY through each wechat group. Finally 132 residents finished both self-assessment and more than 2 pieces of director-assessments. Directors in charge of Gynecologic and Obstetrics were well-trained by the national or provincial director course for Chinese standardized training residency program.

Subjects and participants

This study enrolled 132 out of 147 residents who were trained in Gynecology and Obstetrics of a standardized residency training program and got both self-assessment and director-assessment at November 2022 apt the Qilu Hospital of Shandong University. In total, 47 directors in charge of Gynecology and Obstetrics over the same period were also recruited for our study. All the enrolled residents were categorized into PGY1 to PGY4, according to their seniority.

Data collection

All the self-assessment and director-assessment were collected by internet questionnaire of Wenjuanxing, a network questionnaire website. The residents who had only self-assessments or director-assessment were excluded. Each residents were gathered by Wechat group when they start the by PGY were standardized residency training program in our department of Gynecology and Obstetrics. So it helped to contact graduates who finished the standardized residency training program in Gynecology and Obstetrics.

Sample

In our research, a total of 100 residents trainees were involved, as well as 47 residents graduates. We conducted a cross-sectional study with these 147 residents, and finally 132 residents finished both self-assessment and director-assessment by internet questionnaire of Wenjuanxing, a network questionnaire website. EPAs-evaluation were referred to EPA scale of Peking Union Medical College Hospital, shown in Table 1. The scoring method is shown in Table 2. Table 1. Entrustable professional activities (EPAs) categoriesNumberCategory1Admit a patient2Select and interpret auxiliary examinations3Diagnose and make the differential diagnosis4Make therapeutic decision5Compose medical documents6Report a case7Recognize and manage general clinical conditions8Recognize and manage emergent and critical conditions9Transfer and hand over a patient10Perform informed consent11Perform basic operation12Perform health education13Inform bad news14Perform clinical education15Manage public health eventsTable 2Levels of each entrustable professional activity (EPA)Scale details1Cannot perform certain professional activities as a resident under the direct supervision of a superior physician2Perform certain professional activities with a superior physician together3Perform certain professional activities under the supervision and guidance of a superior physician4Perform certain professional activities without the presence of a superior physician; when help is needed, need the presence of a superior physician to recheck all performances.5Perform certain professional activities without the presence of a superior physician; when help is needed, need the presence of a superior physician to recheck important performances.6Perform certain professional activities without the presence of the superior physician; when help is needed, need the guidance and recheck of superior physician over the phone.7Perform certain professional activities without the need for supervision and guidance from a superior physician.8Can provide supervision and guidance for others in certain professional activities.

Ethical approvals

The study was approved by the Qilu Hospital of Shandong University Institutional Review Board.

Statistics

Statistical software IBM SPSS (version 23.0) was used. All the questionnaires were administered using the Wenjuanxing website (https://www.wjx.cn/). Data was collected by using Excel (Microsoft, Redwood, WA, United States), and statistical analyzed by using SPSS (version 23.0). All the scores of assessments were described in Exp. and Std. Kruskal–Wallis test was used in comparisons between self-assessments and director-assessments for each EPA out of different PGYs. Mann–Whitney U test was used for each EPA assessments between every two different postgraduate years (PGYs). Analysis of PGY and identify position on the EPA scores of director assessments was analyzed by generalized estimated equation (GEE). The p value of 0.05 was considered statistically significant.

Results

The influence of PGYs and training status on EPA scores

Characteristics of residents and questionnaires

In this study, total of 399 questionnaires of assessment were collected, including 132 self-assessment and 267 director-assessment questionnaires. Enrolled residents were categorized into PGY1 to PGY4, according to their seniority. Number and percentage of residents, director-assessment questionnaires, self-assessment were shown in Table 3. For each residents, an average of more than 2 director-assessments were obtained to partly avoid subjective factors. We use the average score of director-assessments for statistical analysis. Table 3. Characteristics of residents and questionnairesCharacteristicsPGY1PGY2PGY3PGY4TotalNumber of residents, n(%)25(18.94%)32(24.24%)39(29.55%)36(27.27%)132Number of director-assessment questionnaires, n(%)52(19.48%)66(24.72%)86(32.21%)63(23.60%)267Number of self-assessment25(18.94%)32(24.24%)39(29.55%)36(27.27%)132Number of director-assessment for each residents (mean,SD)2.85 ± 1.062.80 ± 1.232.60 ± 1.32.48 ± 1.18Number of different position Social training residents11305 Entrusted training residents78111137 Professional master residents1717242179 Permanent staff115411

Higher seniority has a higher EPA score in director-assessment

Date of director-assessment showed that higher seniority has a higher EPA scores (p < 0.05, Table 4 and Fig. 1). In detail, compared to PGY4, there were 6 EPAs of PGY1, 10 EPAs of PGY2 and 1 EPAs of PGY3 had significant difference (p < 0.05, shown in Table 5). Table 4. Scores of director-assessment in different postgraduate years (PGYs)EPAsPGY1PGY2PGY3PGY4TotalChi-squarep valueEPA14.87 ± 1.015.38 ± 0.765.76 ± 0.756.24 ± 0.975.63 ± 0.9827.8950.0001EPA25.04 ± 1.055.45 ± 0.745.79 ± 0.766.06 ± 0.925.64 ± 0.9220.2370.0002EPA35.00 ± 0.925.28 ± 0.645.69 ± 0.836.07 ± 0.945.56 ± 0.9222.1990.0001EPA44.74 ± 0.934.90 ± 0.955.59 ± 0.745.66 ± 1.005.28 ± 0.9826.0660.0001EPA55.19 ± 0.915.73 ± 0.715.95 ± 0.846.35 ± 0.885.86 ± 0.9222.3890.0001EPA65.34 ± 1.005.73 ± 0.815.81 ± 0.876.25 ± 0.875.82 ± 0.9213.6860.0034EPA74.56 ± 0.974.64 ± 0.865.26 ± 0.795.41 ± 1.035.01 ± 0.9719.2110.0002EPA85.20 ± 1.065.47 ± 0.745.86 ± 0.756.29 ± 0.905.76 ± 0.9421.2200.0001EPA95.13 ± 1.055.54 ± 1.035.83 ± 0.806.27 ± 0.925.75 ± 1.0121.8050.0001EPA105.30 ± 1.095.64 ± 0.765.87 ± 0.865.96 ± 1.005.73 ± 0.9510.1760.0171EPA115.54 ± 1.085.72 ± 0.896.10 ± 0.856.39 ± 0.835.98 ± 0.9514.9990.0018EPA125.80 ± 1.295.95 ± 0.816.34 ± 0.896.41 ± 0.846.16 ± 0.9710.6820.0136EPA135.22 ± 1.155.42 ± 0.885.75 ± 1.006.17 ± 0.915.68 ± 1.0316.1280.0011EPA144.79 ± 1.184.86 ± 1.325.50 ± 1.065.74 ± 1.085.28 ± 1.2117.1300.0007EPA154.96 ± 0.994.90 ± 1.255.34 ± 1.075.62 ± 1.185.24 ± 1.1610.2160.0168Fig. 1Comparison of Director-assessment in EPAs of different PGYs between self-assessment and director-assessment. A Comparison of Director-assessment in EPAs of PGY1 between self-assessment and director-assessment. B Comparison of Director-assessment in EPAs of PGY2 between self-assessment and director-assessment. C Comparison of Director-assessment in EPAs of PGY3 between self-assessment and director-assessment. D Comparison of Director-assessment in EPAs of PGY4 between self-assessment and director-assessment. * p < 0.05; **p < 0.01, *p < 0.001Table 5Generalized estimated equation analysis of director-assessment questionnairesEPAsFactorParameterExp(b)95% confidence intervalp valueEPA1PGYPGY10.2550.092 ~ 0.7010.008PGY20.4230.204 ~ 0.8760.021PGY30.6170.329 ~ 1.1580.133PGY4Refer. valuePositionSocial training residents0.3570.212 ~ 0.6030.000Entrusted training residents0.5550.283 ~ 1.0870.086Professional master0.3510.182 ~ 0.6750.002Permanent staffRefer. valueEPA2PGYPGY10.3600.156 ~ 0.8340.017PGY20.5410.226 ~ 1.2970.169PGY30.7590.450 ~ 1.2830.304PGY4Refer. valuePositionSocial training residents0.3280.243 ~ 0.4410.000Entrusted training residents0.5000.321 ~ 0.7770.002Professional master0.3310.222 ~ 0.4950.000Permanent staffRefer. valueEPA3PGYPGY10.3450.141 ~ 0.8430.020PGY20.4540.213 ~ 0.9660.040PGY30.6850.392 ~ 1.1990.185PGY4Refer. valuePositionSocial training residents0.5800.1470.147Entrusted training residents0.7320.3030.303Professional master0.4250.0090.009Permanent staffRefer. valueEPA4PGYPGY10.4010.158 ~ 1.0480.062PGY20.4690.243 ~ 0 .9050.024PGY30.9390.569 ~ 1.5500.805PGY4Refer. valuePositionSocial training residents0.3690.258 ~ 0.5260.000Entrusted training residents0.5400.349 ~ 0.8350.006Professional master0.2950.183 ~ 0.4740.000Permanent staffRefer. valueEPA5PGYPGY10.3130.137 ~ 0.7160.006PGY20.5350.304 ~ 0.9420.030PGY30.6660.419 ~ 1.0580.085PGY4Refer. valuePositionSocial training residents0.3990.236 ~ 0.6740.001Entrusted training residents0.5960.321 ~ 1.1040.100Professional master0.3540.190 ~ 0.6610.001Permanent staffRefer. valueEPA6PGYPGY10.4010.163 ~ 0.9820.046PGY20.5900.334 ~ 1.0430.070PGY30.6420.417 ~ 0.9880.044PGY4Refer. valuePositionSocial training residents0.6200.328 ~ 1.1720.142Entrusted training residents0.6840.359 ~ 1.3030.248Professional master0.3820.199 ~ 0.7350.004Permanent staffRefer. valueEPA7PGYPGY10.4240.161 ~ 1.1200.083PGY20.4580.238 ~ 0.8850.020PGY30.8570.542 ~ 1.3570.511PGY4Refer. valuePositionSocial training residents0.3320.199 ~ 0.5540.000Entrusted training residents0.5780.341 ~ 0.9790.041Professional master0.3270.188 ~ 0.5680.000Permanent staffRefer. valueEPA8PGYPGY10.3350.113 ~ 0.9880.047PGY20.4390.278 ~ 0.6920.000PGY30.6490.392 ~ 1.0740.092PGY4Refer. valuePositionSocial training residents0.3290.182 ~ 0.5950.000Entrusted training residents0.4760.303 ~ 0.7460.001Professional master0.2910.174 ~ 0.4870.000Permanent staffRefer. valueEPA9PGYPGY10.3200.092 ~ 1.1080.083PGY20.4820.219 ~ 1.0610.020PGY30.6450.369 ~ 1.1270.511PGY4Refer. valuePositionSocial training residents0.7120.177 ~ 0.3950.437Entrusted training residents0.4290.302 ~ 0 .6080.000Professional master0.2640.303 ~ 1.6750.000Permanent staffRefer. valueEPA10PGYPGY10.3200.188 ~ 1.3950.191PGY20.7230.422 ~ 1.2400.239PGY30.9090.554 ~ 1.4910.706PGY4Refer. valuePositionSocial training residents0.6110.322 ~ 1.1590.131Entrusted training residents0.4510.290 ~ 0.7000.000Professional master0.2930.198 ~ 0.4350.000Permanent staffRefer. valueEPA11PGYPGY10.4250.149 ~ 1.2150.110PGY20.5090.277 ~ 0.9350.030PGY30.7480.445 ~ 1.2590.275PGY4Refer. valuePositionSocial training residents0.5380.194 ~ 0.4520.151Entrusted training residents0.4850.303 ~ 0.7750.003Professional master0.2960.230 ~ 1.2550.000Permanent staffRefer. valueEPA12PGYPGY10.5450.185 ~ 1.6090.272PGY20.6320.405 ~ 0.9840.042PGY30.9380.635 ~ 1.3870.749PGY4Refer. valuePositionSocial training residents0.7110.334 ~ 1.5160.378Entrusted training residents0.4840.280 ~ 0.8370.009Professional master0.2920.183 ~ 0.4690.000Permanent staffRefer. valueEPA13PGYPGY10.3850.133 ~ 1.1200.080PGY20.4720.219 ~ 1.0170.055PGY30.6570.416 ~ 1.0390.073PGY4Refer. valuePositionSocial training residents1.000.234 ~ 4.3150.995Entrusted training residents0.6160 .331 ~ 1.1460.126Professional master0.4000.218 ~ 0.7350.003Permanent staffRefer. valueEPA14PGYPGY10.3850.491 ~ 1.2620.100PGY20.4120.235 ~ 0.7210.002PGY30.7870.124 ~ 1.2010.320PGY4Refer. valuePositionSocial training residents0.3450.103 ~ 1.1520.084Entrusted training residents0.5330.258 ~ 1.1040.090Professional master0.2980 .169 ~ 0.5260.000Permanent staffRefer. valueEPA15PGYPGY10.5170.169 ~ 1.5830.248PGY20.4860.231 ~ 1.0230.057PGY30.7520.414 ~ 1.3650.248PGY4Refer. valuePositionSocial training residents0.6760.231 ~ 1.9790.475Entrusted training residents0.5690.275 ~ 1.1780.129Professional master0.3020.157 ~ 0.5810.000Permanent staffRefer. value

There was a trend of growth in self-assessment of residents’ training, but no significance was found in statistic of our study (Table 6). Table 6. Scores of self-assessment in different postgraduate years (PGYs)EPAsPGY1PGY2PGY3PGY4TotalChi-squarep valueEPA15.69 ± 1.095.89 ± 0.756.16 ± 1.346.22 ± 1.246.03 ± 1.074.8540.1828EPA25.65 ± 1.025.78 ± 0.706.07 ± 1.225.94 ± 1.195.89 ± 1.094.2510.2356EPA35.53 ± 1.035.70 ± 0.675.86 ± 1.136.08 ± 1.185.82 ± 1.056.1110.1063EPA45.04 ± 1.195.11 ± 1.315.46 ± 1.145.58 ± 1.185.34 ± 1.195.0920.1652EPA55.96 ± 1.156.15 ± 0.776.72 ± 0.986.50 ± 1.366.39 ± 1.129.2980.0256EPA65.69 ± 1.295.96 ± 0.896.33 ± 1.046.36 ± 1.206.13 ± 1.136.8830.0757EPA74.69 ± 1.524.85 ± 1.065.04 ± 1.275.02 ± 1.524.93 ± 1.352.1450.5428EPA85.62 ± 1.105.96 ± 0.906.23 ± 0.906.36 ± 1.056.09 ± 1.009.3630.0248EPA95.46 ± 1.635.56 ± 1.315.93 ± 1.556.03 ± 1.345.79 ± 1.474.02202591EPA105.77 ± 1.146.22 ± 0.936.13 ± 0.995.97 ± 1.536.04 ± 1.181.9920.5741EPA116.08 ± 1.166.33 ± 1.076.51 ± 1.016.51 ± 1.116.39 ± 1.083.0790.3797EPA126.23 ± 1.186.48 ± 1.016.72 ± 0.886.67 ± 1.176.56 ± 1.064.0610.2550EPA135.42 ± 1505.96 ± 1.166.14 ± 1.216.19 ± 1.215.98 ± 1.285.8270.1203EPA145.27 ± 1.315.41 ± 1.555.21 ± 1.665.64 ± 1.175.38 ± 1.440.9820.8055EPA155.27 ± 1.515.15 ± 1.465.02 ± 1.655.17 ± 1.445.13 ± 1.520.2440.9702

Different EPAs seniority had different EPA score

The scores for different EPA projects were different. Among these 15 EPAs, residents got relatively low scores in EPA4 (Make therapeutic decision, total score of 5.28 ± 0.98), EPA7 (Recognize and manage general clinical conditions, total score of 5.01 ± 0.97), EPA14 (Perform clinical education, total score of 5.28 ± 1.21) and EPA15 (Manage public health events, total score of 5.24 ± 1.16). It was shown in Table 4.

Significant difference were found between different position

Among the four different position of residents (social training residents, entrusted training residents, professional master and permanent staff), permanent staff has the highest EPA scores in this EPAs-evaluation study. Compared to permanent staff, 6 EPAs showed significance in social training residents, 8 EPAs showed significant difference in entrusted training residents (p < 0.05). Professional master showed the least desirable EPA scores in all of these 15 EPAs (shown in Table 6 and Fig. 2).Fig. 2. Director-assessment of different position. A Director-assessment of EPAs in permanent staff of different PGYs. B Director-assessment of EPAs in professional master of different PGYs. C Director-assessment of EPAs in entrusted training residents of different PGYs. D Director-assessment of EPAs in social training residents of different PGYs

Significant difference was found in PGYs between self-assessment and director-assessment.

In this study, we found significant difference between self-assessment and director-assessment in PGY1, PGY2 and PGY3 (p < 0.05). In detail, there were 8 EPAs found had significant higher scores of self-assessment than that of director-assessment in PGY2, meanwhile, 6 EPAs in PGY3, and EPAs in PGY1 (Fig. 3). However, for PGY4 residents, self-assessment and director-assessment results were relatively consistent. Self-assessment sores of PGY1 residents were significant higher in 3 EPAs than director-assessment scores, what is more, no significance was found in PGY4 residents (Fig. 3).Fig. 3. Director-assessment of different seniority (different postgraduate years). A Self-assessment of different PGYs. B Statistic of different PGYs in self-assessment productive history of patients. It show the gestation, abortion, and partus maturus. C Director-assessment of different PGYs. D Statistic of different PGYs in director-assessment. * p < 0.05; **p < 0.01, ***p < 0.001

Different EPA evaluation between self- and director- assessment

Compared with director-assessment, significant difference was found in self-assessment on admit a patient, select and interpret auxiliary examinations, compose medical documents, perform basic operation and perform health education (shown in Fig. 3).

Discussion

In China, the standardized training of resident doctors (including the training of clinical medical degree) is an important part of post-graduation medical education [10]. It plays a role in medical school education and resident education, which determines the career development direction of young doctors. Evaluation of residents’ competency has always been the focus and difficulty of standardized training. Kinds of evaluation, including mini-clinical-evaluation exercises (Mini-CEX) and direct observation of procedural skills (DOPS), had been performed for decades [16, 17], however educational impact varies greatly and could be influenced by a few factors [18, 19]. Entrustable professional activities (EPAs) is a new competency model which includes the clinical key familiar behavior for teachers and residents to evaluate the comprehensive ability of residents in daily work. It has been recognized that EPAs can better reflect the abilities of residents [14, 20–22].

In this study, we used Chinese EPA scale, which has been proved its validity in Pediatric [10, 15]. We conduct a cross-sectional study towards residents and instructors in Gynecology and Obstetric. Residents were thought to enhance their clinical professional skills as the training time increased [23]. Our results indeed showed that the clinical competence of residents was gradually enhanced year by year according to whether self-assessment or director-assessment. For the trend of competence growth, an obviously growing peak was found at PGY2, which mean students of PGY2 made great progress during their residential training. Compared to director-assessment, residents of PGY1, PGY2 and PGY3 had more or less over-confidence. But self-assessment of PGY4 residents were almost consistent to director-assessment. This result indicated that residents may make mistakes in performing clinical professional behavior based on overconfidence, which should be paid attention during clinical work.

As director-assessment was much more accurate, we did exact statistics on director-assessment. Among different PGYs, standard deviation of PGY1 was more than that of other PGYs on most of EPAs, except EPA3 (Diagnose and make the differential diagnosis), EPA4 (Make therapeutic decision) and EPA15 (Manage public health events). It indicated that graduated medical students had different ability in those EPAs, however relatively consistent ability in these 3 EPAs above. The statistic date of director-assessment show deficiencies in EPA4 (Make therapeutic decision), EPA7 (Recognize and manage general clinical conditions), EPA11 (Perform clinical education) and EPA12 (Manage public health events), which need to be improved in future resident training.

Among different position of residents, permanent staff has the highest EPA scores in this EPAs-evaluation study. The reason might be that permanent staff had higher education degree, and more stronger sense of responsibility and ownership. What’s more, permanent staff might also have better initiative and exercise opportunity. However, residents of professional master showed the least desirable EPA scores in all of these 15 EPAs, which was contribute to deficiencies in clinical practice. Social training residents and entrusted training residents usually had a period of work experience, which result in a better competency basis. But it finally show similar ability score in residents of social training, entrusted training and professional master, except permanent staff. Nevertheless, of the residents group, the social training and the relative number of staff in our hospital were relatively small, which may have statistical bias.

There were also several limitations in this current study. First, as this is a cross-sectional study, more residents could be involved in future study. Second, the study was a cross-sectional study that residents of different PGYs may have individual differences, which may affects the statistical results. A longitudinal study To survey skill assessment of the same residents during the training time may be better for trend research. However, in our study, however growing trend of EPAs still could be concluded according to EPA scores of differen PGYs. Third, there was partly difference between different directors, which we did detailed pre-assessment training to avoid this. Last, only single centre of Qilu hospital was involved. Multiple centre will be joined in this evaluation research.

There were also several limitations in this current study. First, the PGY1 residents who were involved in this study had different training rotate in the first 3 months, which may lead to a certain difference in both self-assessment and director-assessment. But it will be gradually reduced in subsequent PGYs’ assessments. Second, the directors of each resident are not the same. Although we reduce this bias by including two or more directors for assessment, there was still a certain subjective bias. A more comprehensive and detailed process evaluation system will be carried out in future studies.

Conclusion

In conclusion, for evaluation of residents’ competency, we put forward a novel model of EPA assessment to evaluate the quality of residential training teaching and improve the standardization training process of Chinese resident doctors. Our model of EPA assessment show a good evaluation efficiency, and more easier to handle, indicating a broad prospect.

Supplementary Information

Supplementary Material 1.Supplementary Material 2.