Evaluation of a collaborator objective structured clinical examination (COSCE) in postgraduate medical education

Vernon Curran, Adam Reid, Heidi Coombs, Pam Pike, James Farrell, Ian McPherson, Jennifer O'Dea, Bryan Curtis, Julia Trahey

Abstract


Background: Effective intra- and interprofessional collaboration abilities are necessary for safe and effective medical care, however such roles are often informally taught in postgraduate medical education with lack of opportunity for practice and feedback. The Objective Structured Clinical Encounter (OSCE) is a common approach in medical education. Adaptations of the OSCE have been found useful in the assessment of collaborator competencies amongst interprofessional student groups and assessment of intrinsic roles, like collaboration.

Objective: The purpose of this study was to evaluate the effectiveness of a Collaborator Objective Structured Clinical Encounter (COSCE) as a method of formative assessment on collaborator competencies for postgraduate trainees. 

Methods: This study involved a one group, pretest-posttest evaluation conducted in 2018. PGY1 residents completed a Team Skills Scale immediately before and after COSCE participation, completed an evaluation survey to report satisfaction, and were assessed by facilitators and peer assessors using a COSCE rubric.

Results: Residents reported significant improvement in their pre (N=35) to post-team skills (N=37) scores and an overall positive level of satisfaction with the COSCE experience (N=37/39, 94.9% response rate). Transfer of care skills (e.g., handover) demonstrated the lowest performance scores across all COSCE stations. Peer assessor (N = 204) and facilitator scores (N = 47) also indicated a moderate level of interrelatedness. 

Conclusion: A COSCE is a feasible method of formative assessment, fostering reflection and learning, and providing feedback on collaborator skills early in postgraduate medical education. Peer assessment may also hold promise as a formative assessment method on intra- and interprofessional collaboration.


Keywords


interprofessional education; collaborator; formative assessment

Full Text:

PDF

References


Arora, V., Johnson, J., Lovinger, D., Humphrey, H. J., & Meltzer, D. O. (2005). Communication failures in patient sign-out and suggestions for improvement: A critical incident analysis. Quality and Safety in Health Care, 14(6), 401–407. https://doi.org/10.1136/qshc.2005.015107

Barrington, D. (1998). Student evaluation of an interactive, multidisciplinary clinical learning model. Medical Teacher, 20(6), 530–535. https://doi.org/10.1080/01421599880238

Berger, E. D., Chan, M., Kuper, A., Albert, M., Jenkins, D., Harrison, M., & Harris, I. (2012). The CanMEDs role of collaborator: How is it taught and assessed according to faculty and residents? Paediatrics & Child Health, 17(10), 557–560. https://doi.org/10.1093/pch/17.10.557

Biran, L. A. (1991). Self-assessment and learning through GOSCE (group objective structured clinical examination). Medical Education, 25, 475–479. https://doi.org/10.1111/j.1365-2923.1991.tb00100.x

Borowitz, S. M., Waggoner-Fountain, L. A., Bass, E. J., & Sledd, R. M. (2008). Adequacy of information transferred at resident sign-out (inhospital handover of care): A prospective survey. Quality and Safety in Health Care, 17(1), 6–10. https://doi.org/10.1136/qshc.2006.019273

Chander, B., Kule, R., Baiocco, P., Chokhavatia, S., Kotler, D., Poles, M., Zabar, S., Gillespie, C., Ark, T., & Weinshel, E. (2009). Teaching the competencies: Using objective structured clinical encounters for gastroenterology fellows. Clinical Gastroenterology and Hepatology, 7(5), 509–514.

https://doi.org/10.1016/j.cgh.2008.10.028

Dannefer, E. F., Henson, L. C., Bierer, S. B., Grady-Weliky, T. A., Meldrum, S., Nofziger, A. C., Barclay, C., & Epstein, R. M. (2005). Peer assessment of professional competence. Medical Education, 39, 713–722. https://doi.org/10.1111/j.1365-2929.2005.02193.x

Dwyer, T., Glover, T. S., Kennedy, H. M., Herold, J., Wasserstein, D., Nousiainen, M., Ferguson, P., Wadey, V., Murnaghan, M. L., Leroux, T., Semple, J., Hodges, B., & Ogilvie-Harris, D. (2014). How to assess communication, professionalism, collaboration and the other intrinsic CanMEDS roles in orthopedic residents: Use of an objective structured clinical examination (OSCE). Canadian Journal of Surgery, 57(4), 230–236. https://doi.org/10.1503/cjs.018813

Elliot, D. L., Fields, S. A., Keenen, T. L., Jaffe, A. C., & Toffler, W. L. (1994). Use of a group objective structured clinical examination with first-year medical students. Academic Medicine, 69(12), 990–992. http://doi.org/10.1097/00001888-199412000-00018

Finn, G. M., & Garner, J. (2011). Twelve tips for implementing a successful peer assessment. Medical Teacher, 33(6), 443–446. https://doi.org/10.3109/0142159X.2010.546909

Frank, J. R., Snell, L., & Sherbino, J. (2015). CanMEDS 2015 Physician Competency Framework. Royal College of Physicians and Surgeons of Canada. http://canmeds.royalcollege.ca/en/framework

Glover, T. S., Abbott, C., Oswald, A., & Frank, J. R. (2015). CanMEDS teaching and assessment tools guide. Royal College of Physicians and Surgeons of Canada.

Hall, P., Marshall, D., Weaver, L., Boyle, A., & Taniguchi, A. (2011). A method to enhance student teams in palliative care: Piloting the McMaster-Ottawa team observed structured clinical encounter. Journal of Palliative Medicine, 14(6), 744–750. https://doi.org/10.1089/jpm.2010.0295

Hepburn, K., Tsukuda, R. A., & Fasser, C. (2002). Team skills scale. In G. D. Heinemann & A. M. Zeiss (Eds.), Team performance in health care: Assessment and development (pp. 159–163). Kluwer Academic/Plenum. http://doi.org/10.1007/978-1-4615-0581-5

Jefferies, A., Simmons, B., Tabak, D., McIlroy, J. H., Lee, K. S., Roukema, H., & Skidmore, M. (2007). Using an objective structured clinical examination (OSCE) to assess multiple physician competencies in postgraduate training. Medical Teacher, 29(2/3), 183–191. https://doi.org/10.1080/01421590701302290

Johnson, D. P., Zimmerman, K., Staples, B., McGann, K. A., Frush, K., & Turner, D. A. (2015). Multicenter development, implementation, and patient safety impacts of a simulation-based module to teach handovers to pediatric residents. Hospital Pediatrics, 5(3), 154–158. https://doi.org/10.1542/hpeds.2014-0050

Lofgren, R. P., Gottlieb, D., Williams, R. A., & Rich, E. C. (1990). Post-call transfer of resident responsibility. Journal of General Internal Medicine, 5(6), 501–505. https://doi.org/10.1007/BF02600880

Nofziger, A. C., Naumburg, E. H., Davis, B. J., Mooney, C. J., & Epstein, R. M. (2010). Impact of peer assessment on the professional development of medical students: A qualitative study. Academic Medicine, 85(1), 140–147. https://doi.org/10.1097/ACM.0b013e3181c47a5b

Puddester, D., MacDonald, C. J., Clements, D., Gaffney, J., & Wiesenfeld, L. (2015). Designing faculty development to support the evaluation of resident competency in the intrinsic CanMEDS roles: Practical outcomes of an assessment of program director needs. BMC Medical Education, 15, 100. https://doi.org/10.1186/s12909-015-0375-5

Royal College of Physicians and Surgeons of Canada (2021). Collaborator. Retrieved December 9, 2020, from https://www.royalcollege.ca/rcsite/canmeds/framework/canmeds-role-collaborator-e

Singleton, A., Smith, F., Harris, T., Ross-Harper, R., & Hilton, S. (1999). An evaluation of the team objective structured clinical examination (TOSCE). Medical Education, 33, 34–41. https://doi.org/10.1046/j.1365-2923.1999.00264.x

Solomon, P., Marshall, D., Boyle, A., Burns, S., Casimiro, L. M., Hall, P., & Weaver, L. (2011). Establishing face and content validity of the McMaster-Ottawa team observed structured clinical encounter (TOSCE). Journal of Interprofessional Care, 25(4), 302–304. https://doi.org/10.3109/13561820.2011.571353

Speyer, R., Pilz, W., Van, D. K., & Brunings, J. W. (2011). Reliability and validity of student peer assessment in medical education: A systematic review. Medical Teacher, 33(11), e572–e585. https://doi.org/10.3109/0142159X.2011.610835

Symonds, I., Cullen, L., & Fraser, D. (2003). Evaluation of a formative interprofessional team objective structured clinical examination (ITOSCE): A method of shared learning in maternity education. Medical Teacher, 25(1), 38–41. https://doi.org/10.1080/0142159021000061404

Varkey, P., & Natt, N. (2007). The objective structured clinical examination as an educational tool in patient safety. Joint Commission Journal on Quality and Patient Safety, 33(1), 48–53. https://doi.org/10.1016/s1553-7250(07)33006-7

Warren, A. E., Allen, V. M., Bergin, F., Hazelton, L., Alexiadis-Brown, P., Lightfoot, K., Mcsweeney, J., Singleton, J. F., Sargeant, J., & Mann, K. (2014). Understanding, teaching and assessing the elements of the CanMEDS professional role: Canadian program directors’ views. Medical Teacher, 36(5), 390–402. https://doi.org/10.3109/0142159X.2014.890281

Williams, R., Miler, R., Shah, B., Chokhavatia, S., Poles, M., Zabar, S., Gillespie, C., & Weinshel, E. (2011). Observing handoffs and telephone management in GI fellowship training. The American Journal of Gastroenterology, 106(8), 1410–1414. https://doi.org/10.1038/ajg.2011.107

Wolff, M. J., Balzora, S., Poles, M., Zabar, S., Mintah, A., Wong, L., Weinshel, E., & Malter, L. B. (2015). Objective structured clinical examination as a novel tool in inflammatory bowel disease fellowship education. Inflammatory Bowel Diseases, 21(4), 759–765. https://doi.org/10.1097/MIB.0000000000000317

Zabar, S., Adams, J., Kurland, S., Shaker-Brown, A., Porter, B., Horlick, M., Hanley, K., Altshuler, L., Kalet, A., & Gillespie, C. (2016). Charting a key competency domain: Understanding resident physician interprofessional collaboration (IPC) skills. Journal of General Internal Medicine, 31(8), 846–853. https://doi.org/10.1007/s11606-016-3690-6




DOI: https://doi.org/10.11157/fohpe.v22i1.456

Refbacks

  • There are currently no refbacks.