Using the Freeth/Kirkpatrick model to evaluate interprofessional learning outcomes in a rural setting

Pippa Craig; Sally Hall; Christine Phillips

doi:10.11157/fohpe.v17i1.124

Authors

Pippa Craig University of Notre Dame
Sally Hall ANU Medical School
Christine Phillips Australian National University

DOI:

https://doi.org/10.11157/fohpe.v17i1.124

Keywords:

interprofessional learning, evaluation, rural

Abstract

Background: Considerable claims have been made for the benefits of interprofessional learning (IPL) despite limited evidence of its long-term effectiveness. A collaboration between the Australian National University and the University of South Australia offered opportunities for senior health professional students to undertake IPL teamwork placements in rural NSW—the Health “Hubs and Spokes” Project. The aim of this study was to evaluate, using mixed methods, the outcomes of an IPL project on all four levels of the Freeth/Kirkpatrick evaluation model.

Methods: Students completed a debriefing questionnaire, the Interprofessional Education Perception Scale (IEPS) and the Team Performance Scale (TPS) at the time they undertook IPL placements. A follow-up study measured anticipated professional networks and extent of their interprofessional experience. IPL facilitators, clinical supervisors and local health service representatives were asked to provide their views of perceived project outcomes.

Results: Initial evaluation demonstrated positive student reaction (Level 1), attitude change and collaborative skills (Level 2). While the follow-up study failed to show significant changes in the size of expected professional networks, qualitative exploration of transfer of learning to professional practice (Level 3) and impact of IPL placements in rural locations (Level 4) suggest a positive impact on participants and on healthcare delivery.

Conclusion: The project generated student satisfaction, changes in attitude and skills, and longer-term positive impacts on the community and possibly on the students’ professional practice. This evaluation model can be used effectively to evaluate outcomes of IPL teamwork placements in rural settings and potentially in other locations.

References

Anderson, E. S., & Thorpe, L. N. (2014). Students improve patient care and prepare for professional practice: An interprofessional community-based study. Medical Teacher, 36(6), 495–504.

Australian Medical Council Limited (AMC). (2012). Standards for assessment and accreditation of primary medical programs by the Australian Medical Council. Canberra, Australia: Author.

Australian Nursing & Midwifery Accreditation Council (ANMAC). (2012). Registered nurse accreditation standards. Canberra, Australia: Author.

Australian Pharmacy Council (APC). (2012). Accreditation standards for pharmacy programs in Australia and New Zealand. Canberra, Australia: Author.

Barnes, D., Carpenter, J., & Dickinson, C. (2006). The outcomes of partnerships with mental health service users in interprofessional education: A case study. Health and Social Care in the Community, 14(5), 426–435.

Carpenter, J., Barnes, D., Dickinson, C., & Wooff, D. (2006). Outcomes of interprofessional education for community mental health services in England: The longitudinal evaluation of a postgraduate programme. Journal of Interprofessional Care, 20(2), 145–161.

Chevalier, R. (2004). The link between learning and performance. Performance Improvement, 43(4), 40–44.

Craig, P. L., Barnard, A., Glasgow, N., & May, E. (2014). Evaluating the Health “Hubs and Spokes” interprofessional placements in rural New South Wales, Australia. Journal of Allied Health, 43(3), 176–183.

Craig, P. L., Phillips, C., & Hall, S. (2015). Building social capital with interprofessional student teams in rural settings: A service-learning model. Australian Journal of Rural Health. Advance online publication. doi:10.1111/ajr.12268

Curran, V. R., Sharpe, D., Flynn, K., & Button, P. (2010). A longitudinal study of the effect of an interprofessional education curriculum on student satisfaction and attitudes towards interprofessional teamwork and education. Journal of Interprofessional Care, 24(1), 41–52.

Freeth, D., Hammick, M., Koppel, I., Reeves, S., & Barr, H. (2002). A critical review of evaluations of interprofessional education (Occasional Paper No. 2). October 2002. London, England: Higher Education Academy Health Sciences and Practice Network.

Frye, A. W., & Hemmer, P. A. (2012). Program evaluation models and related theories. Medical Teacher, 34, 288–299.

Fuller, J., Kelly, B., Sartore, G., Fragar, L., Tonna, A., Pollard, G., & Hazell, T. (2007). Use of social network analysis to describe service links for farmers’ mental health. Australian Journal of Rural Health, 15(2), 99–106.

Fuller, J., Kelly, B., Law, S., Pollard, G., & Fragar, L. (2009). Service network analysis for agricultural mental health. BMC Health Services Research, 9, 87.

Furness, P. J., Armitage, H., & Pitt, R. (2011). An evaluation of practice-based interprofessional education initiatives involving service users. Journal of Interprofessional Care, 25, 46–52.

Furness, P. J., Armitage, H., & Pitt, R. (2012). Establishing and facilitating practice- based interprofessional learning: Experiences from the TUILIP project. Nursing Reports, 2, e5.

Gillian, C., Lovrics, E., Halpern, E., Wiljer, D., & Harnett, N. (2011). The evaluation of learner outcomes in interprofessional continuing education: A literature review and an analysis of survey instruments. Medical Teacher, 33, e461–e470.

Hall, P., & Weaver, L. (2001). Interdisciplinary education and teamwork: A long and winding road. Medical Education, 35, 867–875.

Hammick, M., Freeth, D., Koppel, I., Reeves, S., & Barr, H. (2007). A best evidence systematic review of interprofessional education: BEME Guide No. 9. Medical Teacher, 29, 735–751.

Hunter, J., Watt-Watson, J., McGillion, M., Raman-Wilms, L., Cockburn, L., Lax, L., . . . Salter, M. (2008). An interfaculty pain curriculum: Lessons learned from six years’ experience. Pain, 140, 74–86.

Kilminster, S., Hale, C., Lascelles, M., Morris, P., Roberts, T., Stark, P., . . . Thistlethwaite, J. (2004). Learning for real life: Patient-focused interprofessional workshops offer added value. Medical Education, 38, 717–726.

Kirkpatrick, D. (1996). Revising Kirkpatrick’s four-level model. Training & Development, 50(1), 54–59.

Luecht, R. M., Madsen, M. K., Taugher, M. P., & Peterson, B. J. (1990). Assessing professional perceptions: Design and validation of an interprofessional education perception scale. Journal of Allied Health, 19, 181–191.

Mann, K., Sargeant, J., & Hill, T. (2009a). Knowledge translation in interprofessional education: What difference does interprofessional education make to practice? Learning in Health and Social Care, 8(3), 154–164.

Mann, K., McFetridge-Durdle, J., Martin-Misener, R., Clovis, J., Rowe, R., Beanlands, H., Sarria, M. (2009b). Interprofessional education for students of the health professions: The “Seamless Care” model. Journal of Interprofessional Care, 23(3), 224–233.

Marin, A., & Hampton, K. (2007). Simplifying the personal network name generator: Alternatives to traditional multiple and single name generators. Field Methods, 19(2),163–193.

O’Carroll, V., Braid, M., Ker, J., & Jackson, C. (2012). How can student experience enhance the development of a model of interprofessional clinical skills education in the practice placement setting? Journal of Interprofessional Care, 26, 508–510.

Parker, K. (2013). A better hammer in a better toolbox: Considerations for the future of programme evaluation. Medical Education, 47(5), 440–442.

Parker, V., McNeil, K., Higgins, I., Mitchell, R., Paliadelis, P., Giles, M., & Parmenter, G. (2013). How health professionals conceive and construct interprofessional practice in rural settings: A qualitative study. BMC Health Service Research, 13, 500. Retrieved from http://www.biomedcentral.com/1472-6963/13/500

Reeves, S., Zwarenstein, M., Goldman, J., Barr, H., Freeth, D., Hammick, M., & Koppel, I. (2008). Interprofessional education: Effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews, 2008(1). doi:10.1002/14651858.CD002213.pub2

Reeves, S., Perrier, L., Goldman, J., Freeth, D., & Zwarenstein, M. (2013). Interprofessional education: Effects on professional practice and healthcare outcomes [update]. Cochrane Database of Systematic Reviews, 2013(3). doi:10.1002/14651858.CD002213.pub3

Thistlethwaite, J., Kumar, K., Moran, M., Saunders, R., & Carr, S. (2015). Journal of Interprofessional Care, 29(4), 292–297.

Thompson, B. M., Levine, R. E., Kennedy, F., Naik, A. D., Foldes, C. A., Coverdale, J. H., . . . Haidet, P. (2009). Evaluating the quality of learning-team processes

in medical education: Development and validation of a new measure. Academic Medicine, 84(Suppl. 10), S124–127.

Van Der Gaag, M., & Snijders, T. A. B. (2005). The resource generator: Social capital quantification with concrete items. Social Networks, 27(1), 1–29.

Zwarenstein, M., Atkins, J., Hammick, M., Barr, H., Koppel, I., & Reeves, S. (1999). Interprofessional education and systematic review: A new initiative in evaluation. Journal of Interprofessional Care, 13, 417–424.