Quasi-Arithmetic Scoring and Group Project Marking

P H Vossen, Suraj Ajit, Andrew Dean

Research output: Contribution to Book/ReportConference Contribution

Abstract

Quasi-Arithmetic Scoring Theory grew out of a desire to justify peer assessment as a meaningful educational measurement technique. Peer assessment is used to evaluate group work and other collaborative learning activities. Students evaluate each other’s participation to joint (project) assignments. The evaluations are used to split an overall team score into individual student scores to inform student grading. So far, a formal treatment of peer assessment methodology has been lacking.
Traditionally, peer assessment uses bounded scales (equivalent to the standard percentage scale) together with a single n-ary operation: the weighted arithmetic mean. This practice is rather weak and questionable. Quasi-Arithmetic Scoring Theory puts peer assessment and judgmental scales in general on a sound and strong mathematical foundation. The bounded scales will be equipped with two basic operations (different from their arithmetic counterparts) so that they get the structure of a module. Peer assessment is then modelled by bivariate module theory of one bounded scale of peer ratings acting on another bounded scale of student scores.
A bounded scale is an interval [푚푖푛, 푚푎푥] of scores 푠 such that 푚푖푛 ≤ 푠 ≤ 푚푎푥 together with quasiarithmetic operations of addition and scalar multiplication of scores, where scores may be multidimensional composites, i.e. consisting of subscores (items or criteria). Such bounded scales are mathematical structures called modules. Modules have the right structure for educational metrics and related statistics. The percentage scale [0,1] plays the role of standard bounded scale. All properties valid for the standard scale hold for all bounded scales, e.g. the popular Likert scales.
To model peer assessment, Quasi-Arithmetic Scoring Theory needs to be extended to bivariate modules, i.e., a bounded scale of peer ratings acting upon another bounded scale of student scores. The assessor sets a group score as the default student score. Also, he specifies a scoring rule to define the action of peer ratings on student scores. The action of peer ratings on scores can be constrained to a subscale around the group score so that unrealistic deviations from the group score will be avoided.
Moreover, the impact of peer ratings on scores can be made weaker or stronger
There exist three distinct types of scoring rules for peer assessment. Besides the traditional arithmetic type of scoring rule, there are two types of scoring rules using modules with addition, scalar multiplication and quasi-arithmetic mean as main operations. The traditional approach is deceptively simple requiring
nothing more than knowledge of arithmetic and its related concept of arithmetic average – but it is weak and inadequate, resting on questionable assumptions about the calculus of scores. The modern approach based on modules. distinguishes between quasi-additive and quasi-multiplicative scoring rules,
using either one of the two module operations. All three scoring rules come with several extensions which allow fine-tuning of the underlying scoring models to tutor preferences, course specifics, or university regulations.
Original languageEnglish
Title of host publicationProceeedings of the 12th Annual International Conference of Education, Research and Innovation, iCERi 2019
Number of pages11
Publication statusAccepted/In press - 29 Sep 2019
Event12th annual International Conference of Education, Research and Innovation - Seville, Spain
Duration: 12 Nov 201913 Nov 2019

Conference

Conference12th annual International Conference of Education, Research and Innovation
CountrySpain
CitySeville
Period12/11/1913/11/19

Fingerprint

project group
rating
student
Group
group work
grading
tutor

Cite this

Vossen, P. H., Ajit, S., & Dean, A. (Accepted/In press). Quasi-Arithmetic Scoring and Group Project Marking. In Proceeedings of the 12th Annual International Conference of Education, Research and Innovation, iCERi 2019
Vossen, P H ; Ajit, Suraj ; Dean, Andrew. / Quasi-Arithmetic Scoring and Group Project Marking. Proceeedings of the 12th Annual International Conference of Education, Research and Innovation, iCERi 2019. 2019.
@inproceedings{e1a8cace6c5342969e4c6bf71d247713,
title = "Quasi-Arithmetic Scoring and Group Project Marking",
abstract = "Quasi-Arithmetic Scoring Theory grew out of a desire to justify peer assessment as a meaningful educational measurement technique. Peer assessment is used to evaluate group work and other collaborative learning activities. Students evaluate each other’s participation to joint (project) assignments. The evaluations are used to split an overall team score into individual student scores to inform student grading. So far, a formal treatment of peer assessment methodology has been lacking.Traditionally, peer assessment uses bounded scales (equivalent to the standard percentage scale) together with a single n-ary operation: the weighted arithmetic mean. This practice is rather weak and questionable. Quasi-Arithmetic Scoring Theory puts peer assessment and judgmental scales in general on a sound and strong mathematical foundation. The bounded scales will be equipped with two basic operations (different from their arithmetic counterparts) so that they get the structure of a module. Peer assessment is then modelled by bivariate module theory of one bounded scale of peer ratings acting on another bounded scale of student scores.A bounded scale is an interval [푚푖푛, 푚푎푥] of scores 푠 such that 푚푖푛 ≤ 푠 ≤ 푚푎푥 together with quasiarithmetic operations of addition and scalar multiplication of scores, where scores may be multidimensional composites, i.e. consisting of subscores (items or criteria). Such bounded scales are mathematical structures called modules. Modules have the right structure for educational metrics and related statistics. The percentage scale [0,1] plays the role of standard bounded scale. All properties valid for the standard scale hold for all bounded scales, e.g. the popular Likert scales.To model peer assessment, Quasi-Arithmetic Scoring Theory needs to be extended to bivariate modules, i.e., a bounded scale of peer ratings acting upon another bounded scale of student scores. The assessor sets a group score as the default student score. Also, he specifies a scoring rule to define the action of peer ratings on student scores. The action of peer ratings on scores can be constrained to a subscale around the group score so that unrealistic deviations from the group score will be avoided.Moreover, the impact of peer ratings on scores can be made weaker or strongerThere exist three distinct types of scoring rules for peer assessment. Besides the traditional arithmetic type of scoring rule, there are two types of scoring rules using modules with addition, scalar multiplication and quasi-arithmetic mean as main operations. The traditional approach is deceptively simple requiringnothing more than knowledge of arithmetic and its related concept of arithmetic average – but it is weak and inadequate, resting on questionable assumptions about the calculus of scores. The modern approach based on modules. distinguishes between quasi-additive and quasi-multiplicative scoring rules,using either one of the two module operations. All three scoring rules come with several extensions which allow fine-tuning of the underlying scoring models to tutor preferences, course specifics, or university regulations.",
author = "Vossen, {P H} and Suraj Ajit and Andrew Dean",
year = "2019",
month = "9",
day = "29",
language = "English",
booktitle = "Proceeedings of the 12th Annual International Conference of Education, Research and Innovation, iCERi 2019",

}

Vossen, PH, Ajit, S & Dean, A 2019, Quasi-Arithmetic Scoring and Group Project Marking. in Proceeedings of the 12th Annual International Conference of Education, Research and Innovation, iCERi 2019. 12th annual International Conference of Education, Research and Innovation, Seville, Spain, 12/11/19.

Quasi-Arithmetic Scoring and Group Project Marking. / Vossen, P H; Ajit, Suraj; Dean, Andrew.

Proceeedings of the 12th Annual International Conference of Education, Research and Innovation, iCERi 2019. 2019.

Research output: Contribution to Book/ReportConference Contribution

TY - GEN

T1 - Quasi-Arithmetic Scoring and Group Project Marking

AU - Vossen, P H

AU - Ajit, Suraj

AU - Dean, Andrew

PY - 2019/9/29

Y1 - 2019/9/29

N2 - Quasi-Arithmetic Scoring Theory grew out of a desire to justify peer assessment as a meaningful educational measurement technique. Peer assessment is used to evaluate group work and other collaborative learning activities. Students evaluate each other’s participation to joint (project) assignments. The evaluations are used to split an overall team score into individual student scores to inform student grading. So far, a formal treatment of peer assessment methodology has been lacking.Traditionally, peer assessment uses bounded scales (equivalent to the standard percentage scale) together with a single n-ary operation: the weighted arithmetic mean. This practice is rather weak and questionable. Quasi-Arithmetic Scoring Theory puts peer assessment and judgmental scales in general on a sound and strong mathematical foundation. The bounded scales will be equipped with two basic operations (different from their arithmetic counterparts) so that they get the structure of a module. Peer assessment is then modelled by bivariate module theory of one bounded scale of peer ratings acting on another bounded scale of student scores.A bounded scale is an interval [푚푖푛, 푚푎푥] of scores 푠 such that 푚푖푛 ≤ 푠 ≤ 푚푎푥 together with quasiarithmetic operations of addition and scalar multiplication of scores, where scores may be multidimensional composites, i.e. consisting of subscores (items or criteria). Such bounded scales are mathematical structures called modules. Modules have the right structure for educational metrics and related statistics. The percentage scale [0,1] plays the role of standard bounded scale. All properties valid for the standard scale hold for all bounded scales, e.g. the popular Likert scales.To model peer assessment, Quasi-Arithmetic Scoring Theory needs to be extended to bivariate modules, i.e., a bounded scale of peer ratings acting upon another bounded scale of student scores. The assessor sets a group score as the default student score. Also, he specifies a scoring rule to define the action of peer ratings on student scores. The action of peer ratings on scores can be constrained to a subscale around the group score so that unrealistic deviations from the group score will be avoided.Moreover, the impact of peer ratings on scores can be made weaker or strongerThere exist three distinct types of scoring rules for peer assessment. Besides the traditional arithmetic type of scoring rule, there are two types of scoring rules using modules with addition, scalar multiplication and quasi-arithmetic mean as main operations. The traditional approach is deceptively simple requiringnothing more than knowledge of arithmetic and its related concept of arithmetic average – but it is weak and inadequate, resting on questionable assumptions about the calculus of scores. The modern approach based on modules. distinguishes between quasi-additive and quasi-multiplicative scoring rules,using either one of the two module operations. All three scoring rules come with several extensions which allow fine-tuning of the underlying scoring models to tutor preferences, course specifics, or university regulations.

AB - Quasi-Arithmetic Scoring Theory grew out of a desire to justify peer assessment as a meaningful educational measurement technique. Peer assessment is used to evaluate group work and other collaborative learning activities. Students evaluate each other’s participation to joint (project) assignments. The evaluations are used to split an overall team score into individual student scores to inform student grading. So far, a formal treatment of peer assessment methodology has been lacking.Traditionally, peer assessment uses bounded scales (equivalent to the standard percentage scale) together with a single n-ary operation: the weighted arithmetic mean. This practice is rather weak and questionable. Quasi-Arithmetic Scoring Theory puts peer assessment and judgmental scales in general on a sound and strong mathematical foundation. The bounded scales will be equipped with two basic operations (different from their arithmetic counterparts) so that they get the structure of a module. Peer assessment is then modelled by bivariate module theory of one bounded scale of peer ratings acting on another bounded scale of student scores.A bounded scale is an interval [푚푖푛, 푚푎푥] of scores 푠 such that 푚푖푛 ≤ 푠 ≤ 푚푎푥 together with quasiarithmetic operations of addition and scalar multiplication of scores, where scores may be multidimensional composites, i.e. consisting of subscores (items or criteria). Such bounded scales are mathematical structures called modules. Modules have the right structure for educational metrics and related statistics. The percentage scale [0,1] plays the role of standard bounded scale. All properties valid for the standard scale hold for all bounded scales, e.g. the popular Likert scales.To model peer assessment, Quasi-Arithmetic Scoring Theory needs to be extended to bivariate modules, i.e., a bounded scale of peer ratings acting upon another bounded scale of student scores. The assessor sets a group score as the default student score. Also, he specifies a scoring rule to define the action of peer ratings on student scores. The action of peer ratings on scores can be constrained to a subscale around the group score so that unrealistic deviations from the group score will be avoided.Moreover, the impact of peer ratings on scores can be made weaker or strongerThere exist three distinct types of scoring rules for peer assessment. Besides the traditional arithmetic type of scoring rule, there are two types of scoring rules using modules with addition, scalar multiplication and quasi-arithmetic mean as main operations. The traditional approach is deceptively simple requiringnothing more than knowledge of arithmetic and its related concept of arithmetic average – but it is weak and inadequate, resting on questionable assumptions about the calculus of scores. The modern approach based on modules. distinguishes between quasi-additive and quasi-multiplicative scoring rules,using either one of the two module operations. All three scoring rules come with several extensions which allow fine-tuning of the underlying scoring models to tutor preferences, course specifics, or university regulations.

M3 - Conference Contribution

BT - Proceeedings of the 12th Annual International Conference of Education, Research and Innovation, iCERi 2019

ER -

Vossen PH, Ajit S, Dean A. Quasi-Arithmetic Scoring and Group Project Marking. In Proceeedings of the 12th Annual International Conference of Education, Research and Innovation, iCERi 2019. 2019