Peer Grading Tool PGT

Note for Lecturers: If you are interested in using PGT, please contact author (see below)

» version information

1 General
In group work, ideally all members contribute equally to the overall success of the project, but usually in individually different ways. Peer grading is used to evaluate individual contributions in group work ([Kaufman et.al. 1999], [Brutus & Donia 2010]), in our context the online tool PGT is used for this purpose. Each group member grades all other group members (peer grading) as well as himself/herself (self-grading) according to definable criteria and receives an evaluation in the form of an anonymized comparison. The evaluations are done with a point scale according to definable weighted criteria.

A lecturer (resp. teacher or instructor) can configure a peer grading with regard to the number and designation of the criteria, their weighting and the point scale, as well as the start and end date for the period of the . After the peer grading , the lecturer receives a mail notification with all individual and group evaluations. This allows the lecturer to compare his assessment of the group contribution of individual group members with the peer evaluations.

Both the instructor and the participants can use their respective evaluations to reflect on the teaching and learning and benefit from them for further group work.

2 Other Peer Grading Tools
Comparable peer grading tools are either only commercially available with a licensing model (e.g., CATME from Purdue University), or have limitations such as requiring participants to have a Google account (peer grading tool as a plugin for Google Spreadsheets), which does not go along with the idea of an open software system.

The Moodle activity "Workshop" is similar to the desired peer grading scenario. However, analysis has shown that essential aspects that are realized in the PGT tool regarding functionality, level of detail, flexibility and customization are not available. Thus, the PGT tool was implemented in December 2022 as an in-house development at the Dep. of Business of the Bern Univ. of Applied Sciences (BUAS) with Angular technology resulting in a web based application, funded by the BFH E-Learning Förderprogramm.

3 Functionality
a) Creation of a peer grading process by the lecturer incl. import of participants in shape of a (UTF-8) csv file with participant name, (optional) matriculation number, email address, and group number.



b) Each group member receives an email with a link to the individualized peer grading sheet and fills it out completely (input values are validated).

c) Upon completion of the peer grading , all participants and the instructor receive evaluations via email.

PG-Review-Everdeen.PNG

In this example, the peer grading is based on 4 criteria with points 1..8. The documentation with the meaning of the criteria and the point scales is outside the PGT system and is to be made known to the participants by other means (see example for criteria catalog below). The evaluations of one's own contributions by other group members are anonymized to the extent that only the mean value of all third-party evaluations is given and thus exact conclusions about evaluating persons are not possible.

The group average is calculated by averaging all third-party assessments within a group. For each group member, the deviation between the group average and the average of the third-party assessments of the individual group member can be determined from this. In the evaluation section, the difference between self-assessment and third-party assessment is also shown, which enables critical reflection also with regard to future teamwork. The group average is calculated by averaging all third-party assessments within a group.

d) The lecturer has control over the peer grading process at any time:

abb-2-2_overview_after-invitations-sent.png

4 Criteria catalog
It is by decision not presented by the PGT system, since it is usually available as a semi-structured document and would have to be imported or entered in a time-consuming complex manner. Here is an example from the course Business Case Studies" (BCS) in the Department of Business at the Bern University of Applied Sciences. It is presented in the study guide of the course. The standard score for a "good" assessment is set at 5/6 points.


Criterion "teamwork":
8/7: Exceptional contribution to the group process; "doer"; 
     proactively advances project goals; is extremely committed;    
     "sacrifices" himself/herself for project success; 
     accomplishes much more than expected; opinion leader; 
     high communication skills.
6/5: Committed member, contribution not too high, not too low; 
     participates in the group process with an average sense of responsibility; 
     has an integrating effect in conflicts.
4/3: Provides contributions on demand; neutral attitude towards project success; 
     does not make motivational advances when there are "sags" in the team.
2/1: Unproductive to counterproductive attitude; 
     does not contribute to the success of the project or hinders/prevents its progress. 
     Destructive charisma.

Criterion "Quality of contributions":
8/7: Well above the expected quality.
6/5: Corresponds to the expected quality.
4/3: Less than the expected quality.
2/1: Insufficient quality.

Criterion "Quantity of contributions":
8/7: Well above the expected quantity.
6/5: Corresponds to the expected quantity.
4/3: Less than the expected quantity.
2/1: Insufficient quantity.

Criterion "Reliability":
8/7: Takes appointments seriously, actively seeks substitutes or alternatives 
     if unable to attend or misses appointments; offers substitutes for what 
     has been missed; ensures that the overall plan is adhered to, 
     provides corrective impetus if necessary.
6/5: Mostly reliable, but adapts to possibly fluctuating team attitude; 
     little pronounced striving for workload compensation 
     if (e.g. after absence) there is a low allocation of tasks; 
     hints at possible deficits in the project are given from time to time.
4/3: Multiple absences or failure to complete tasks; no or late sign-off; 
     only in rare cases are indications of possible deficits in the project 
     to be expected.
2/1: Unreliable, can hardly be entrusted with tasks critical to success; 
     (almost) no communication recognizable in case of failures.

Literature


Author: Eduard Klein / Feb 29, 2024