Evaluation of a collaborator objective structured clinical examination (COSCE) in postgraduate medical education
DOI:
https://doi.org/10.11157/fohpe.v22i1.456Keywords:
interprofessional education, collaborator, formative assessmentAbstract
Background: Effective intra- and interprofessional collaboration abilities are necessary for safe and effective medical care, however such roles are often informally taught in postgraduate medical education with lack of opportunity for practice and feedback. The Objective Structured Clinical Encounter (OSCE) is a common approach in medical education. Adaptations of the OSCE have been found useful in the assessment of collaborator competencies amongst interprofessional student groups and assessment of intrinsic roles, like collaboration.
Objective: The purpose of this study was to evaluate the effectiveness of a Collaborator Objective Structured Clinical Encounter (COSCE) as a method of formative assessment on collaborator competencies for postgraduate trainees.
Methods: This study involved a one group, pretest-posttest evaluation conducted in 2018. PGY1 residents completed a Team Skills Scale immediately before and after COSCE participation, completed an evaluation survey to report satisfaction, and were assessed by facilitators and peer assessors using a COSCE rubric.
Results: Residents reported significant improvement in their pre (N=35) to post-team skills (N=37) scores and an overall positive level of satisfaction with the COSCE experience (N=37/39, 94.9% response rate). Transfer of care skills (e.g., handover) demonstrated the lowest performance scores across all COSCE stations. Peer assessor (N = 204) and facilitator scores (N = 47) also indicated a moderate level of interrelatedness.
Conclusion: A COSCE is a feasible method of formative assessment, fostering reflection and learning, and providing feedback on collaborator skills early in postgraduate medical education. Peer assessment may also hold promise as a formative assessment method on intra- and interprofessional collaboration.
References
Arora, V., Johnson, J., Lovinger, D., Humphrey, H. J., & Meltzer, D. O. (2005). Communication failures in patient sign-out and suggestions for improvement: A critical incident analysis. Quality and Safety in Health Care, 14(6), 401–407. https://doi.org/10.1136/qshc.2005.015107
Barrington, D. (1998). Student evaluation of an interactive, multidisciplinary clinical learning model. Medical Teacher, 20(6), 530–535. https://doi.org/10.1080/01421599880238
Berger, E. D., Chan, M., Kuper, A., Albert, M., Jenkins, D., Harrison, M., & Harris, I. (2012). The CanMEDs role of collaborator: How is it taught and assessed according to faculty and residents? Paediatrics & Child Health, 17(10), 557–560. https://doi.org/10.1093/pch/17.10.557
Biran, L. A. (1991). Self-assessment and learning through GOSCE (group objective structured clinical examination). Medical Education, 25, 475–479. https://doi.org/10.1111/j.1365-2923.1991.tb00100.x
Borowitz, S. M., Waggoner-Fountain, L. A., Bass, E. J., & Sledd, R. M. (2008). Adequacy of information transferred at resident sign-out (inhospital handover of care): A prospective survey. Quality and Safety in Health Care, 17(1), 6–10. https://doi.org/10.1136/qshc.2006.019273
Chander, B., Kule, R., Baiocco, P., Chokhavatia, S., Kotler, D., Poles, M., Zabar, S., Gillespie, C., Ark, T., & Weinshel, E. (2009). Teaching the competencies: Using objective structured clinical encounters for gastroenterology fellows. Clinical Gastroenterology and Hepatology, 7(5), 509–514.
https://doi.org/10.1016/j.cgh.2008.10.028
Dannefer, E. F., Henson, L. C., Bierer, S. B., Grady-Weliky, T. A., Meldrum, S., Nofziger, A. C., Barclay, C., & Epstein, R. M. (2005). Peer assessment of professional competence. Medical Education, 39, 713–722. https://doi.org/10.1111/j.1365-2929.2005.02193.x
Dwyer, T., Glover, T. S., Kennedy, H. M., Herold, J., Wasserstein, D., Nousiainen, M., Ferguson, P., Wadey, V., Murnaghan, M. L., Leroux, T., Semple, J., Hodges, B., & Ogilvie-Harris, D. (2014). How to assess communication, professionalism, collaboration and the other intrinsic CanMEDS roles in orthopedic residents: Use of an objective structured clinical examination (OSCE). Canadian Journal of Surgery, 57(4), 230–236. https://doi.org/10.1503/cjs.018813
Elliot, D. L., Fields, S. A., Keenen, T. L., Jaffe, A. C., & Toffler, W. L. (1994). Use of a group objective structured clinical examination with first-year medical students. Academic Medicine, 69(12), 990–992. http://doi.org/10.1097/00001888-199412000-00018
Finn, G. M., & Garner, J. (2011). Twelve tips for implementing a successful peer assessment. Medical Teacher, 33(6), 443–446. https://doi.org/10.3109/0142159X.2010.546909
Frank, J. R., Snell, L., & Sherbino, J. (2015). CanMEDS 2015 Physician Competency Framework. Royal College of Physicians and Surgeons of Canada. http://canmeds.royalcollege.ca/en/framework
Glover, T. S., Abbott, C., Oswald, A., & Frank, J. R. (2015). CanMEDS teaching and assessment tools guide. Royal College of Physicians and Surgeons of Canada.
Hall, P., Marshall, D., Weaver, L., Boyle, A., & Taniguchi, A. (2011). A method to enhance student teams in palliative care: Piloting the McMaster-Ottawa team observed structured clinical encounter. Journal of Palliative Medicine, 14(6), 744–750. https://doi.org/10.1089/jpm.2010.0295
Hepburn, K., Tsukuda, R. A., & Fasser, C. (2002). Team skills scale. In G. D. Heinemann & A. M. Zeiss (Eds.), Team performance in health care: Assessment and development (pp. 159–163). Kluwer Academic/Plenum. http://doi.org/10.1007/978-1-4615-0581-5
Jefferies, A., Simmons, B., Tabak, D., McIlroy, J. H., Lee, K. S., Roukema, H., & Skidmore, M. (2007). Using an objective structured clinical examination (OSCE) to assess multiple physician competencies in postgraduate training. Medical Teacher, 29(2/3), 183–191. https://doi.org/10.1080/01421590701302290
Johnson, D. P., Zimmerman, K., Staples, B., McGann, K. A., Frush, K., & Turner, D. A. (2015). Multicenter development, implementation, and patient safety impacts of a simulation-based module to teach handovers to pediatric residents. Hospital Pediatrics, 5(3), 154–158. https://doi.org/10.1542/hpeds.2014-0050
Lofgren, R. P., Gottlieb, D., Williams, R. A., & Rich, E. C. (1990). Post-call transfer of resident responsibility. Journal of General Internal Medicine, 5(6), 501–505. https://doi.org/10.1007/BF02600880
Nofziger, A. C., Naumburg, E. H., Davis, B. J., Mooney, C. J., & Epstein, R. M. (2010). Impact of peer assessment on the professional development of medical students: A qualitative study. Academic Medicine, 85(1), 140–147. https://doi.org/10.1097/ACM.0b013e3181c47a5b
Puddester, D., MacDonald, C. J., Clements, D., Gaffney, J., & Wiesenfeld, L. (2015). Designing faculty development to support the evaluation of resident competency in the intrinsic CanMEDS roles: Practical outcomes of an assessment of program director needs. BMC Medical Education, 15, 100. https://doi.org/10.1186/s12909-015-0375-5
Royal College of Physicians and Surgeons of Canada (2021). Collaborator. Retrieved December 9, 2020, from https://www.royalcollege.ca/rcsite/canmeds/framework/canmeds-role-collaborator-e
Singleton, A., Smith, F., Harris, T., Ross-Harper, R., & Hilton, S. (1999). An evaluation of the team objective structured clinical examination (TOSCE). Medical Education, 33, 34–41. https://doi.org/10.1046/j.1365-2923.1999.00264.x
Solomon, P., Marshall, D., Boyle, A., Burns, S., Casimiro, L. M., Hall, P., & Weaver, L. (2011). Establishing face and content validity of the McMaster-Ottawa team observed structured clinical encounter (TOSCE). Journal of Interprofessional Care, 25(4), 302–304. https://doi.org/10.3109/13561820.2011.571353
Speyer, R., Pilz, W., Van, D. K., & Brunings, J. W. (2011). Reliability and validity of student peer assessment in medical education: A systematic review. Medical Teacher, 33(11), e572–e585. https://doi.org/10.3109/0142159X.2011.610835
Symonds, I., Cullen, L., & Fraser, D. (2003). Evaluation of a formative interprofessional team objective structured clinical examination (ITOSCE): A method of shared learning in maternity education. Medical Teacher, 25(1), 38–41. https://doi.org/10.1080/0142159021000061404
Varkey, P., & Natt, N. (2007). The objective structured clinical examination as an educational tool in patient safety. Joint Commission Journal on Quality and Patient Safety, 33(1), 48–53. https://doi.org/10.1016/s1553-7250(07)33006-7
Warren, A. E., Allen, V. M., Bergin, F., Hazelton, L., Alexiadis-Brown, P., Lightfoot, K., Mcsweeney, J., Singleton, J. F., Sargeant, J., & Mann, K. (2014). Understanding, teaching and assessing the elements of the CanMEDS professional role: Canadian program directors’ views. Medical Teacher, 36(5), 390–402. https://doi.org/10.3109/0142159X.2014.890281
Williams, R., Miler, R., Shah, B., Chokhavatia, S., Poles, M., Zabar, S., Gillespie, C., & Weinshel, E. (2011). Observing handoffs and telephone management in GI fellowship training. The American Journal of Gastroenterology, 106(8), 1410–1414. https://doi.org/10.1038/ajg.2011.107
Wolff, M. J., Balzora, S., Poles, M., Zabar, S., Mintah, A., Wong, L., Weinshel, E., & Malter, L. B. (2015). Objective structured clinical examination as a novel tool in inflammatory bowel disease fellowship education. Inflammatory Bowel Diseases, 21(4), 759–765. https://doi.org/10.1097/MIB.0000000000000317
Zabar, S., Adams, J., Kurland, S., Shaker-Brown, A., Porter, B., Horlick, M., Hanley, K., Altshuler, L., Kalet, A., & Gillespie, C. (2016). Charting a key competency domain: Understanding resident physician interprofessional collaboration (IPC) skills. Journal of General Internal Medicine, 31(8), 846–853. https://doi.org/10.1007/s11606-016-3690-6
Downloads
Published
How to Cite
Issue
Section
License
On acceptance for publication in FoHPE the copyright of the manuscript is signed over to ANZAHPE, the publisher of FoHPE.
