Lily Zeng, Teaching and Learning Innovation Centre (TALIC)
Summative peer review of teaching allows colleagues with substantial knowledge of teaching to provide meaningful triangulation of student evaluations of teaching (Chism, 2007). It is a widespread practice in higher education institutions across America, Australia, Mainland China, the UK (Smith, 2012). The University of Hong Kong rolled out this policy recently. It serves to inform personnel decisions such as promotion, contract renewal, tenure, or merit pay. However, it is not uncommon for teachers to exhibit resistance or hesitation towards summative peer review of teaching.
Compared to developmental peer review of teaching, teachers’ hesitation about summative peer review often stems mainly from two concerns: the power hierarchy arising from the appointment of reviewers and the validity and reliability of peer review (Zeng, 2020).
The appointment of reviewers often has implications for the relationship between the reviewers and the reviewees. In a developmental or collaborative model of peer review of teaching, knowledge of the subject area, expertise in teaching and pedagogy, and the key stakeholder of teaching (e.g. students) are perceived as key resources to enhance teachers’ development. The reviewers are therefore often peers from similar subject fields, teachers with more teaching experience or those with a proven track-record in teaching. A hierarchy of expertise is common in the selection of reviewers.
However, university teachers more often perceive a hierarchy of power than hierarchy of expertise when it comes to the summative peer review, as higher levels of seniority often serve as a key selection criterion for reviewers. Expertise in teaching and pedagogy among teachers on teaching-focused positions is not necessarily considered higher levels of seniority, as the teaching track is often regarded as inferior to the academic track in many universities.
Parallel to reviewees concerns about power hierarchy, reviewers also often report hesitations because they have to provide reports that will significantly impact their colleagues’ careers. If there is limited training for the reviewers, their confidence in their ability to conduct a valid review can be undermined. These factors can create a sense of alienation or hesitance among both reviewees and reviewers (Gosling, 2014).
Teachers’ concerns regarding the validity and reliability of summative peer review include potential bias due to limited snapshots of a teacher’s performance (Kell & Annetts, 2009), the subjective interpretations of individual reviewers (Shortland, 2010), and the absence of standardized and validated instruments (Thomas, Chie, Abraham, Raj, & Beh, 2014).
To implement summative peer review of teaching effectively, it is advisable to develop standardized and validated instruments to minimize the subjectivity of reviewers’ interpretations, thereby enhancing the validity and reliability of the review process. Following effective assessment principles, clear criteria can support consistency of ratings. A well-designed instrument provides reviewers with an easy-to-use tool and instils greater confidence among reviewers and reviewees. To promote reliability of the reviews, it is crucial to provide training workshops for the reviewers-to-be, helping them understand and apply the criteria before official reviews. It might also be a good idea to run calibration workshops regularly as a continuing support to the reviewers, although this has resource implications. Well-designed instruments also potentially encourage guidance for teachers on how to enhance student learning outcomes. Hopefully, this can lead to long-term impacts in promoting teaching excellence within the university.
To enhance validity, it might be beneficial to include evidence-based effective practices and widely accepted teaching excellence frameworks in the higher education context, such as the U21 Teaching Standards Framework (U21 Educational Innovation Steering Group, 2018) and the Professional Standards Framework (AdvanceHE, 2023). The appointment as peer reviewers of teachers who have demonstrated evidence of expertise in teaching and pedagogy (e.g. teaching excellence award winners, senior fellows of AdvanceHE) might also improve the validity of the reviews. These individuals have a major career focus on teaching, may be more familiar with aspects of teaching excellence, and might be well-motivated to carry out peer review.
To prepare for summative peer review, individual reviewees may consider inviting reviewers or peers to conduct a prior developmental review using the official instrument provided by the faculty or university. This can help establish formative peer review of teaching processes before dealing with summative peer review.
References
Advance HE. (2023). Professional Standards Framework for teaching and supporting learning in higher education 2023. Retrieved March 5, 2023, from https://s3.eu-west-2.amazonaws.com/assets.creode.advancehe-document-manager/documents/advance-he/PSF%202023%20-%20Screen%20Reader%20Compatible%20-%20final_1675089549.pdf
Chism, N. V. N. (2007). Peer review of teaching: A sourcebook (2nd ed.). Anker Publishing.
Gosling, D. (2014). Collaborative peer-supported review of teaching. In J. Sachs & M. Parsell (Eds.), Peer review of learning and teaching in higher education (pp. 13-31). Springer. https://doi.org/10.1007/978-94-007-7639-5
Kell, C., & Annetts, S. (2009). Peer review of teaching embedded practice or policy-holding complacency? [Article]. Innovations in Education & Teaching International, 46(1), 61-70. https://doi.org/10.1080/14703290802646156
Shortland, S. (2010). Feedback within peer observation: continuing professional development and unexpected consequences. Innovations in Education and Teaching International, 47(3), 295-304. https://doi.org/10.1080/14703297.2010.498181
Smith, H. (2012). The unintended consequences of grading teaching [Article]. Teaching in Higher Education, 17(6), 747-754. https://doi.org/10.1080/13562517.2012.744437
Thomas, S., Chie, Q. T., Abraham, M., Raj, S. J., & Beh, L.-S. (2014). A qualitative review of literature on peer review of teaching in higher education: An application of the SWOT framework. Review of Educational Research, 84(1), 112-159. https://doi.org/http://dx.doi.org/10.3102/0034654313499617
U21 Educational Innovation Steering Group. (2018). U21 Teaching Standards Framework – Full Description. Retrieved March 2, 2023, from https://universitas21.com/sites/default/files/2018-03/U21-TSF-Full-Description-MASTER.pdf
Zeng, L. M. (2020). Peer review of teaching in higher education: A systematic review of its impact on the professional development of university teachers from the teaching expertise perspective. Educational Research Review, 31, 100333. https://doi.org/10.1016/j.edurev.2020.100333
Interesting post, thanks Lily. I can see a lot of advantages in having a dedicated team of experts in teaching excellence as a trained cadre of reviewers. This could include SFHEA or PFHEA, Winners of awards for teaching excellence, Associate Deans for Teaching and Learning and the like. Deploying research professors because of their seniority seems less logical in my view. I wonder what lessons can be drawn from experiences in other universities. I remember Geoff Crisp doing impressive work in this area in the past, including at UNSW. “For a summative peer review of teaching program to be successful, peer reviewers must be trained and experienced.”
https://ltr.edu.au/resources/PP6-46_Peer%20review%20of%20teaching%20for%20promotion%20purposes%20Adelaide%202009.pdf
I would be delighted to learn from colleagues who have experience or expertise in summative peer review of teaching. I am conducting my first observation later this month & I aim to be descriptive, more than judgmental.
Thank you, David, for the comments. Dr. Anna Rowe from UNSW was invited to share their PRT practices in a previous event organised by TALIC. During the event, I asked her a question about the availability of sufficient eligible reviewers who were willing to participate in the reviews. She agreed that this was indeed a challenging aspect of their review scheme.
I guess adopting a descriptive approach in PRT is beneficial. It promotes the objective description of what is observed, minimizing the influence of subjectivity in the review process. If the description is well aligned with the official instrument, it will help enhance the overall consistency and reliability of the assessment process.