The Challenge Of Peer Assessment in Higher Education

Peer Assessment (PA) in higher education (HE), i.e., asking students to assess each other to inform teacher’s own assessment of their learning progress and contribution to groupwork, is neither trivial nor is it the king’s road to accurate, unbiased educational assessment. However, granted a deep understanding by both teacher and students of its strengths and weaknesses, opportunities and threats, PA can be the method of choice in cases where classical assessment is impossible or may be grossly misleading, inefficient, or incomplete.

We will propose a new assessment framework for a combined Group-Peer Assessment (GPA) of the kind of assignments referred to above. Our goal is to promote awareness and appreciation among (preservice) teachers of the unique features and many advantages of GPA when conducted correctly and properly. The paper will be based on our experiences with GPA using a paper-based approach, and on the development of interactive software tools to support teachers and students when conducting PA or GPA. We will not discuss the rise of online GPA in MOOC, as the challenges therein are of a rather technical and procedural nature, while our assessment framework is focussed on more fundamental, much-neglected aspects of measurement & computation with a focus on validity, flexibility, and utility.
We will describe a unified family of peer rating, student scoring and group grading models in HE. This still growing family of models is called system 𝓠 and consists of:
(1)a collection of bounded scales (such as the percentage scale) with adjustable neutral elements and equipped with quasi-arithmetic operations of addition, subtraction, inverting, and scalar multiplication of ratings, scores, or grades by non-negative decimal numbers.
(2)a collection of formulae for peer and student ratings as well as for student contributions and scores of three types: linear, rational, and exponential, corresponding to three types of constrained percentage scales.
(3)a collection of formulae for the quasi-arithmetic means of student ratings, contributions, and scores, that correctly represent the overall group work and its outcomes and guarantee that the average of student scores equals the group score (called Split-Join-Invariance).
The backbone of 𝓠 is the concept of bounded scale equipped with a module structure. The traditional approaches to PA have uncritically adopted a statistical view of measurement and scaling. This has led to some paradoxical results when it comes to merging the peer ratings of students (with a focus on group dynamics and process quality) with the teacher-assigned scores of group work (with a focus on group outcomes and product quality). It is well-known that adding and multiplying multiple ratings or scores without taking the boundedness of the scales into account may lead to results outside the accepted domain of ratings or scores (e.g., percentages). Suggested repair mechanisms (e.g., using ad hoc parameters, applying min and max operators, or simply capping) don’t do justice to the original collected data nor do they satisfy the teachers and students.

These problems can be solved by adopting a quasi-arithmetic calculus that redefines addition and multiplication of ratings or scores so that they make sense on bounded scales: one can add, subtract, and invert ratings, scores, and grades without ever leaving the adopted bounded scale. Moreover, the quasi-arithmetic mean of ratings, scores and grades on bounded scales can be defined and applied in place of the arithmetic mean that is more suitable for unbounded scales.

Since ICERI2019, we have generalized, streamlined, and improved our assessment framework in several ways. However, the most significant innovation is a new scoring model with remarkable properties: the double-exponential scoring rule.