Optimizing short-term in-class process evaluation: analyzing the effectiveness of teaching interventions in pharmaceutical education using repeated measures analysis

TL;DR

This study evaluates how teaching interventions improve pharmacy students' understanding of the PCNE Classification System in China using statistical analysis.

Contribution

The study introduces the use of Repeated Measures Analysis to assess teaching interventions in pharmaceutical education.

Findings

Repeated Measures Analysis showed significant improvements in understanding the PCNE system over time.

Pearson correlation confirmed a strong link between understanding levels and the likelihood of using the PCNE system.

Homework scores validated the questionnaire's accuracy in measuring understanding.

Abstract

The assessment of the effectiveness of teaching interventions in enhancing students’ understanding of the Pharmaceutical Care Network Europe (PCNE) Classification System is crucial in pharmaceutical education. This is especially true in regions like China, where the integration of the PCNE system into undergraduate teaching is limited, despite its recognized benefits in addressing drug-related problems in clinical pharmacy practice. Therefore, this study aimed to evaluate the effectiveness of teaching interventions in improving students’ understanding of the PCNE Classification System in pharmaceutical education.

Undergraduate pharmacy students participated in a series of sessions focused on the PCNE system, including lectures (t1), case analyses (t2), and practical implementation (t3). The levels of understanding were evaluated using time-course questionnaires. Initially, paired samples t-Tests were used to compare understanding levels between different time points. Subsequently, Repeated Measures Analysis (RMA) was employed. Pearson correlation analysis was conducted to examine the relationship between understanding levels and the usability and likelihood of using the PCNE system, as reported in the questionnaires.

The paired samples t-Tests indicated insignificant differences between t2 and t3, suggesting limited improvement following the practical implementation of the PCNE system. However, RMA revealed significant time effects on understanding levels in effective respondents and the focused subgroup without prior experience (random intercept models: all p < 0.001; random slope models: all p < 0.001). These results confirmed the effectiveness of all three teaching interventions. Pearson correlation analysis demonstrated significant positive correlations between understanding levels and the usability and likelihood of using the PCNE system at all examined time points. This finding highlighted the reliability of the understanding levels reported in the questionnaires. The homework scores were used as external calibration standards, providing robust external validation of the questionnaire’s validity.

The implementation of RMA provided robust evidence of the positive impact of time on understanding levels. This affirmed the effectiveness of all teaching interventions in enhancing students’ comprehension of the PCNE Classification System. By utilizing RMA, potential errors inherent in common statistical methods, such as t-Tests, were mitigated. This ensured a more comprehensive and accurate assessment of the effectiveness of the teaching interventions.

The online version contains supplementary material available at 10.1186/s12909-024-05754-y.