Script concordance test examinations: Student perception and study approaches
DOI:
https://doi.org/10.11157/fohpe.v20i2.321Keywords:
Script Concordance Test, SCT, Multiple Choice Question, MCQ, ConsequencesAbstract
Introduction: Script concordance test questions (SCTs) are a style of question that has been developed based upon the script theory of information storage and retrieval. SCTs use script theory to examine a clinical reasoning construct purported to be different from that assessed by multiple-choice questions (MCQs). Evidence regarding the construct validity of SCT is somewhat limited, but generally supportive.
Methods: This project involved a content-matched MCQ and SCT practice examination delivered to senior medical students (n = 211) from a single institution with an accompanying survey regarding educational consequences of SCT and MCQ examinations.
Results:Students’ responses differed significantly between how they would prepare for an MCQ examination as compared to an SCT examination. These differences were present in a number of areas, including greater focus on textbooks and use of websites for MCQ examination preparation (p < 0.001). Students reported that for an SCT examination they would benefit most from in-person teaching from local consultants involved in the process of SCT generation. Students felt that they would also benefit from further instruction regarding key elements of SCT technique, and from opportunities to provide written explanations of their reasoning during the exam.
Conclusions:The results suggest that SCTs have educational consequences different from MCQs. Further research into the educational consequences of SCTs, and how these consequences may differ from that of other methods of assessment, is warranted.
References
Charlin, B., Boshuizen, H. P., Custers, E. J., & Feltovich, P. J. (2007). Scripts and clinical reasoning. Med Educ, 41(12), 1178-1184. doi: 10.1111/j.1365-2923.2007.02924.x
Charlin, B., Brailovsky, C., Leduc, C., & Blouin, D. (1998). The Diagnosis Script Questionnaire: A New Tool to Assess a Specific Dimension of Clinical Competence. Advances in Health Sciences Education, 3(1), 51-58.
Charlin, B., Gagnon, R., Pelletier, J., Coletti, M., Abi-Rizk, G., Nasr, C., . . . van der Vleuten, C. (2006). Assessment of clinical reasoning in the context of uncertainty: the effect of variability within the reference panel. Med Educ, 40(9), 848-854. doi: 10.1111/j.1365-2929.2006.02541.x
Charlin, B., Roy, L., Brailovsky, C., Goulet, F., & van der Vleuten, C. (2000). The Script Concordance test: a tool to assess the reflective clinician. Teach Learn Med, 12(4), 189-195. doi: 10.1207/S15328015TLM1204_5
Charlin, B., & van der Vleuten, C. (2004). Standardized assessment of reasoning in contexts of uncertainty: the script concordance approach. Eval Health Prof, 27(3), 304-319. doi: 10.1177/0163278704267043
Cook, D. A., Zendejas, B., Hamstra, S. J., Hatala, R., & Brydges, R. (2014). What counts as validity evidence? Examples and prevalence in a systematic review of simulation-based assessment. Adv Health Sci Educ Theory Pract, 19(2), 233-250. doi: 10.1007/s10459-013-9458-4
Duggan, P., & Charlin, B. (2012). Summative assessment of 5th year medical students’ clinical reasoning by script concordance test: requirements and challenges. BMC Med Educ, 12(1).
Duggan, P., & Charlin, B. (2016). Introducing recent medical graduates as members of Script Concordance Test expert reference panels: what impact? AMEE MedEdPublish.
Fournier, J. P., Demeester, A., & Charlin, B. (2008). Script concordance tests: guidelines for construction. BMC Med Inform Decis Mak, 8, 18. doi: 10.1186/1472-6947-8-18
Gagnon, R., Charlin, B., Coletti, M., Sauve, E., & van der Vleuten, C. (2005). Assessment in the context of uncertainty: how many members are needed on the panel of reference of a script concordance test? Med Educ, 39(3), 284-291. doi: 10.1111/j.1365-2929.2005.02092.x
Hornos, E. H., Pleguezuelos, E. M., Brailovsky, C. A., Harillo, L. D., Dory, V., & Charlin, B. (2013). The practicum script concordance test: an online continuing professional development format to foster reflection on clinical practice. J Contin Educ Health Prof, 33(1), 59-66. doi: 10.1002/chp.21166
Labelle, M., Beaulieu, M., Paquette, D., Fournier, C., Bessette, L., Choquette, D., . . . Thivierge, R. L. (2004). An integrated approach to improving appropriate use of anti-inflammatory medication in the treatment of osteoarthritis in Quebec (Canada): the CURATA model. Med Teach, 26(5), 463-470. doi: 10.1080/0142159042000218669
Lubarsky, S., Charlin, B., Cook, D. A., Chalk, C., & van der Vleuten, C. P. (2011). Script concordance testing: a review of published validity evidence. Med Educ, 45(4), 329-338. doi: 10.1111/j.1365-2923.2010.03863.x
Lubarsky, S., Dory, V., Audétat, M., Custers, E., & Charlin, B. (2015). Using script theory to cultivate illness script formation and clinical reasoning in health professions education. Canadian Medical Education, 6(2), 61-70.
Lubarsky, S., Dory, V., Duggan, P., Gagnon, R., & Charlin, B. (2013). Script concordance testing: from theory to practice: AMEE guide no. 75. Med Teach, 35(3), 184-193. doi: 10.3109/0142159X.2013.760036
Lubarsky, S., Gagnon, R., & Charlin, B. (2011). Script concordance test item response process: The argument for probability versus typicality. Advances in Health Sciences Education, 17(1), 11-13. doi: 10.1007/s10459-011-9338-8
Messick, S. (1989). Validity. In: Linn RL, editor. Educational Measurement, 3rd Ed, New York: American Council on Education and Macmillan.
Petrella, R. J., & Davis, P. (2007). Improving management of musculoskeletal disorders in primary care: the Joint Adventures Program. Clin Rheumatol, 26(7), 1061-1066. doi: 10.1007/s10067-006-0446-4
Sullivan, G. M., & Artino, A. R., Jr. (2013). Analyzing and interpreting data from likert-type scales. J Grad Med Educ, 5(4), 541-542. doi: 10.4300/JGME-5-4-18
Van Der Vleuten, C. (1996). The assessment of professional competence: developments, research and practical implications. Advances in Health Sciences Education, 1(1), 41-67.
Downloads
Published
How to Cite
Issue
Section
License
On acceptance for publication in FoHPE the copyright of the manuscript is signed over to ANZAHPE, the publisher of FoHPE.
