Keywords

Objective Structured Clinical Examination (OSCE); Pharmacy student prescription; Self-care; OTC (nonprescription medications); Consultation; Counseling; PPCP; QuEST-Scholar; Active learning; In-Person OSCE; Virtual OSCE

Introduction

At the West Coast University School of Pharmacy, the onground PharmD program is a four-year comprehensive pharmacy program with all In-Person activities that include didactics, laboratories, OSCEs, co-curricular, and Interprofessional Educations (IPEs). In contrast, the hybrid PharmD program follows a three-year accelerated format, combining online didactic instructions with scheduled In-Person immersive session that includes exams, hands-on laboratory work, and co-curricular activities. While on-campus classes offer several advantages, hybrid learning also provides unique benefits [1,2]. One key advantage of the hybrid program is its ability to accommodate geographically diverse students by allowing them to enroll and participate remotely. Through online platforms such as Zoom and class technologies, students attend classes from various locations, enabling flexibility and ensuring that the students meet the program’s learning outcomes. Hybrid delivery allows students to manage their schedules more effectively by reducing the need for frequent travel to campus and maintain a healthier balance between academic and personal responsibilities.

The Pharmacy Practice Foundation I (PPFI) course is a 4 credit (60 hours) core course offered in the first term of the program. This course introduces the students to the foundation of pharmacy practice and provides training for students to apply theoretical knowledge in real-life scenarios [3]. The OSCE is a key skills-based assessment in pharmacy education, commonly integrated into foundation courses to assess students’ preparedness for Introductory Pharmacy Practice Experience (IPPE) [4].

There are three main objectives of OSCE assessment in the first-year foundation course:

• OSCE serves as a peak point of the knowledge and skills students have gained in the didactic curriculum.

• It thoroughly prepares students with reviews and assessments before commencing their IPPEs.

• It serves as a crucial assessment point to adapt to evolving educational standards [5,6].

In the PPFI course, the OSCE performance accounted for 10% of the course grade for the 4-year on-ground PharmD program, and 15% in the hybrid PharmD program students. In both programs, the OSCEs constituted a substantial portion of the grade, underscoring the importance of this skills-based assessment in preparing students for their first IPPE.

The COVID-19 pandemic caused unprecedented global disruption, severely limiting In-Person assessments. In response, medical educators globally pursued alternative ways to assess and certify students and trainees to meet the acute demand for healthcare workers. 6 Video conference platforms have become more prominent in the literature evaluating educational methods in health professional students. For instance, Danielle et al. evaluated the satisfaction of examiners and candidates in a virtual OSCE (vOSCE) conducted for graduating Canadian urology residents.

The study reported high satisfaction levels among both groups, highlighting the effectiveness of using a teleconference format for a mock OSCE [7]. In their 2024 study on the transition to Virtual Objective Structured Clinical Examinations (OSCE), Saeed et al. clearly established that both faculty and students held positive views about this new format. Their results demonstrated a high level of overall satisfaction with the Virtual OSCE, with the majority of attendees affirming that they would confidently recommend it to friends or colleagues [8].

At the West Coast University School of Pharmacy, in the PPFI course, the students have learned on

• Role and practice-scope of community pharmacists and pharmacy as well community pharmacy operation.

• Foundational communication skills and theory including structured interview QuEST and pharmacist patient-centered processes.

• Self-Care topics including pain and fever, cold and allergic rhinitis, ophthalmic disorders, otic and oral health, and women’s health.

• The basic drug information of the top 200 prescribed medications.

• APhA Pharmacy-Based Immunization Delivery Certificate training.

In order to create a seamless process for any OSCEs, early planning and training were implemented. Self-Care and nonprescription Over the Counter (OTC) product therapeutics are taught in the Pharmacy Practice Foundation courses (i.e., PPFI and II) as one of the fundamental knowledge and skill areas. The top 200 prescription medications are introduced in this course series. Given that the students will start their IPPE rotation in the community pharmacy after completing their first semester based on the program curriculum, the OSCE is designed with two stations including station I – Prescription consultation and station II – Self-care OTC counseling to prepare the students’ abilities to take good care of patients with pharmacists, technicians, and clerks.

Materials and Methods

Study participants

Two cohorts of first-year pharmacy students nearing the completion of their first semester were included in this study. The first cohort consisted of students enrolled in a 3-year hybrid PharmD program who participated in a virtual Objective Structured Clinical Examination (OSCE). The second cohort included students enrolled in a 4-year on-campus PharmD program who completed the OSCE in person.

OSCE design

Both cohorts completed a structured, two-station OSCE as part of their Pharmacy Practice Foundations I course. The OSCE was designed to assess practical skills in two distinct areas: Station I: Prescription Counseling where students were required to counsel a patient on a new prescription. Evaluations were conducted using a rubric based on the Pharmacists’ Patient Care Process (PPCP). Station II: Self-Care OTC consultation where students conducted a self-care consultation for an Over-the- Counter (OTC) product using the QuEST and SCHOLAR processes.

OSCE procedure

For each station, students were provided with a case scenario with three minutes to review either the prescription or self-care script, a patient interaction with seven minutes to perform the consultation with a Standardized Patient (SP) and five minutes of constructive feedback provided by faculty evaluators and the SP.

Survey feedback on OSCE formats

A post-survey was distributed to collect feedback from students, Standardized Patients (SPs), and faculty evaluators about their experiences with the virtual and in-person OSCEs. The survey was created using Microsoft Forms and disseminated via e-mail. It included close-ended questions with Likert-scale response options (e.g., strongly disagree to strongly agree) to assess perceptions of the OSCE format.

To ensure the clarity and relevance of the survey questions, a pilot test was conducted with 1–2 participants. Feedback from the pilot test informed adjustments to the final survey distributed to the participants.

Statistical analysis

Descriptive statistics, including mean, median, and standard deviation, were calculated for OSCE scores to summarize cohort performance. Survey responses were analyzed descriptively to identify themes and trends in the perceived advantages and challenges of virtual and in-person OSCEs. This feedback was used to develop recommendations for optimizing the virtual OSCE format in the 3-year hybrid PharmD program.

Statistical analysis of OSCE performance

To evaluate differences in student performance between the virtual and in-person OSCE formats, multiple statistical tests were conducted. The data was first analyzed using Julius software to test for normal distribution (using the Shapiro-Wilk test) and variance homogeneity (using Levene's test). Based on these results: For data that did not meet normality or homogeneity assumptions, a non-parametric comparison using the Mann-Whitney U test was employed to assess differences in performance between the two cohorts. Statistical significance for all tests was defined as a p-value of <0.05.

Results

The study population included students (hybrid and onground), evaluators, and standardized patients. Forty-three percent of the students were students who participated in the hybrid cohort and fifty-seven percent of the students participated in the on-ground cohort.

Descriptive statistics summarizing overall OSCE scores of Virtual and In-Person OSCEs are listed in Table 1.

Table 1: Descriptive statistics of overall OSCE scores (Virtual and In-Person).

Normality and variance testing

Shapiro-Wilk tests revealed that virtual overall OSCE scores (p=0.043) was non-normally distributed, while In-Person overall OSCE scores demonstrated a normal distribution (p=0.288).

Comparative analysis

Non-parametric Mann-Whitney U tests found no statistically significant differences between Virtual and In-Person modalities for overall OSCE scores (p=0.179).

As to the score distribution trends, the box plot visualizations (Figure 1) revealed the following trends: Both Virtual and In- Person assessments had similar median scores, with overlapping interquartile ranges. The Virtual scores demonstrated slightly greater variability. The outliers were observed in both modalities, predominantly at the lower end of the score range. These findings indicate comparable performance between Virtual and In-Person modalities for overall OSCE scores. While Virtual scores exhibited slightly higher variability, the lack of significant differences suggests that both assessment modalities are equivalent in measuring student performance.

Figure 1: Overall OSCE score distribution box plot.

Post-survey respondent’s summary

Out of 78 approached students, 51 completed the questionnaire, resulting in a 65.4% response rate. The respondents included 74.5% students (n=38), 13.7% evaluators (n=7), and 11.8% standardized patients (n=6). Table 2 presents the responses of students, evaluators, and standardized patients regarding their perceptions of Virtual versus In-Person OSCE. Most participants either somewhat agreed or strongly when asked about various aspects, such as preparation instructions, overall experience, the demonstration and evaluation of communication and clinical skills, and interactions with standardized patients in both Virtual and In-Person formats.

Table 2: OSCE evaluation form survey.

Communication skills assessments and method of OSCE delivery: Figure 2 shows that most respondents were satisfied with the demonstration and evaluation of communication skills in the OSCEs. Specifically, 30 respondents (58.8%) somewhat agreed or strongly agreed about their satisfaction with Virtual OSCEs, compared to 24 respondents (47.1%) for In-Person OSCEs. The data indicates a generally positive perception of both Virtual and In-Person OSCEs among respondents (Figure 2).

Figure 2: Communication skills assessment in OSCE formats comparison of Virtual vs. In-Person OSCE effectiveness.

The survey questioned the perceptions of the students, standardized patients and students on the effectiveness of orientation and preparation, the assessment method, communication skills evaluation and effectiveness of clinical skills in both Virtual and In-Person OSCE sessions. A summary of the positive responses is presented in Figure 3 below. In general, students strongly and moderately agreed with the effectiveness of both OSCE delivery methods. There was a generally higher positive response in the Virtual OSCEs, yet none were statistically significant.

Figure 3: Comparison of Virtual vs. In-Person OSCE Metrics (Combined “somewhat agree” and “strongly agree” responses.

Discussion

The statistical analysis revealed that both Virtual and In- Person training had similar impact on the performance of firstyear pharmacy students in prescription and self-care patient consultations, as indicated by p-values exceeding the threshold of 0.05. This finding aligns with similar studies, e.g., Gortney et al., found no significant differences between Virtual and In- Person OSCEs regarding student performances. Even though the score of students participating in Virtual OSCE was slightly higher than their peers in In-Person OSCE, the overall assessment of student performances showed no statistically significant difference (p-value>0.05). The results of OSCE assessments suggest that, despite similar performance in specific consultation stations, Virtual OSCE might offer advantages in the evaluation of student competencies. A possible explanation for the slight score difference is that students engaging in the Virtual OSCE from the comfort of their homes might experience less anxiety and fewer distractions compared to those in the In-Person OSCE, who are exposed to the pressures of a classroom environment and the presence of fellow students. These findings imply that while specific skill areas may not differ significantly between the two formats, the Virtual OSCE could provide a more effective overall assessment experience by minimizing external stressors and distractions.

When comparing the evaluations of Virtual and In-Person OSCEs through post-survey feedback, the perceptions of both formats were similar, with no significant overall difference reported by students, standardized patients, and evaluators. However, Virtual OSCEs were rated more favorably in several areas such as preparation instructions, positive experience, and the demonstration and evaluation of communication and clinical skills. This suggests that the Virtual format may offer a more streamlined and less stressful preparation process, as well as potentially providing a more comprehensive assessment of student skills. Similarly, Wollen et al. observed that Virtual OSCEs led to a reduction in assessment time compared to In-Person OSCEs, which shortened the time commitment required from evaluators, IT team members, and standardized patients [9].

Conversely, In-Person OSCEs were rated more positively for the quality of interactions with standardized patients in a realworld clinical setting. Wollen et al. noted that a limitation of Virtual OSCEs is the difficulty in verifying if students are making eye contact with the patient on screen, which raises concerns about academic honesty and the potential for accessing prohibited materials. Therefore, while Virtual OSCEs excel in terms of organization and efficiency, In-Person OSCEs are preferable for ensuring authentic patient interactions and maintaining academic integrity.

Ultimately, both formats have their strengths; the choice between Virtual and In-Person OSCEs should be guided by the specific objectives of the assessment and the aspects of student performance that are most important to evaluate.

OSCEs are integral to pharmacy education, serving as a bridge between theoretical knowledge and practical application. Studies by Bond et al. and Beccaria et al. reinforce their role as reliable and valid tools for competency assessment in pharmacy education. Incorporating Virtual OSCEs into hybrid programs offers accessible solutions, particularly in post-pandemic education models. The findings of this study support the integration of Virtual OSCEs as part of a flexible and comprehensive assessment strategy in pharmacy curricula.

Limitations

One challenge in Virtual OSCE was a technical problem, for example, one student had an internet issue during the Virtual OSCE performance. Another limitation of this research study was partial response rate in the survey portion. There were 51 respondents out of 78 survey recipients who were participants in the OSCEs including students, standardized patients, and evaluators. The partial response rate was 65.4%, which may result in response bias and nonresponse bias. As a result, this can hinder some true differences between Virtual and In-Person OSCE formats.

Conclusion

This study demonstrates that both Virtual and In-Person OSCE formats can effectively assess first-year pharmacy students' clinical skills and knowledge and evaluate student competencies, with no significant differences in performance and perception. The positive feedback on the Virtual OSCE highlights their efficiency and convenience, while In-Person OSCE continue to offer valuable real-world interactions. The findings suggest that the choice of OSCE format should be based on specific assessment goals and context, and environmental reality. In future research, we should address the challenge of internet stability to enhance Virtual OSCE experience. Moreover, we may also utilize Artificial Intelligence (AI) tools in both Virtual and In- Person OSCE formats.

To optimize OSCE delivery, future studies should explore hybrid models that integrate the efficiency of virtual formats with the practicality of in-person encounters. Investigating the long-term impact of virtual OSCE formats on clinical preparedness and patient care outcomes is also essential.

References

1. McCleary-Gaddy A, Yu ET, Spears RD (2022) In-Person, remote, or hybrid instruction? A quality improvement assessment of a six week interprofessional education pathway program for undergraduate pre-health students. Healthcare 10:2399

   [Crossref] [Google Scholar] [PubMed]

2. Singh J, Steele K, Singh L (2021) Combining the best of online and face-to-face learning: Hybrid and blended learning approach for COVID-19, post vaccine, and post-pandemic world. J Educ Technol Syst 50:140-171

   [Google Scholar]

3. Gortney JS, Fava JP, Berti AD, Stewart B (2022) Comparison of student pharmacists' performance on In-Person vs. Virtual OSCEs in a pre-APPE capstone course. Curr Pharm Teach Learn 14:1116-1121

   [Crossref] [Google Scholar] [PubMed]

4. Song YK, Chung EK, Lee YS, Yoon JH, Kim H (2023) Objective structured clinical examination as a competency assessment tool of students' readiness for advanced pharmacy practice experiences in South Korea: a pilot study. BMC Med Educ 23:231

   [Crossref] [Google Scholar] [PubMed]

5. Lim AS, Ling YL (2024) What’s been trending with OSCEs in pharmacy education over the last 20 years? A bibliometric review and content analysis. Curr Pharm Teach Learn 16:212-220

   [Crossref] [Google Scholar] [PubMed]

6. Chan SCC, Choa G, Kelly J, Maru D, Rashid MA (2023) Implementation of Virtual OSCE in health professions education: A systematic review. Med Educ 57:833-843

   [Crossref] [Google Scholar] [PubMed]

7. Jenkins D, Nashed JY, Touma NJ (2023) Virtual OSCE examinations during COVID-19 A 360 satisfaction assessment from examiners and candidates. Can Urol Assoc J 17:E315-E318

   [Crossref] [Google Scholar] [PubMed]

8. Saeed E, Hamad MH, Alhuzaimi AN (2024) Virtual objective structured clinical examination (OSCE) training in the pandemic era: Feasibility, satisfaction, and the road ahead. Cureus 16:e61564

   [Crossref] [Google Scholar] [PubMed]

9. Wollen JT, Gee JS, Nguyen KA, Surati DD (2023) Development of a communication-based Virtual patient counseling objective structured clinical examination (OSCE) for first year student pharmacists. PEC Innov 3:100215

   [Crossref] [Google Scholar] [PubMed]