How do we know that we are assessing the “right things”?

Abstract

 Introduction: As assessment is perceived as a powerful tool for learning, it is important to ensure that assessments reflect the learning considered most significant. Assessment blueprinting offers the opportunity to ensure perceptions of what should be assessed align with what is assessed. 

Methods: An expert panel was asked to determine the percentage of undergraduate assessment that should be devoted to broad domain areas previously agreed to define the outcomes of an undergraduate curriculum; this provided the blueprint. Staff who co-ordinated, implemented and assessed students on clinical runs indicated the percentage of their assessments allocated to the domain areas. Staff who constructed end-of-year summative assessments also analysed their assessments in terms of the domain areas. The “expert panel” blueprint values were then compared using Mann–Whitney U with the actual assessment conducted to determine variations between the ideal and actual assessment. 

Results: What was considered important to assess closely aligned with what was assessed in most domain areas. The exceptions were the underassessment of two domains, Māori Health and Research and Information Literacy.

Conclusions: The chosen methodology identified areas that were under-represented in the actual student assessments. This prompted the school to consider whether this under-representation is problematic; if so, whether to redistribute or increase assessment, and whether the required increases should occur in-course or at the end of the year.