A Comparison of Peer- and Tutor-Grading of an Introductory R Coding Assessment

We investigate the level of agreement between tutor and peer grading of an introductory R programming assessment. Comparing peer and tutor grades we find a strong correlation of 0.848, 95%CI = (0.809, 0.880). Using standard multivariate data analysis techniques we find that tutors and peers grade si...

Full description

Saved in:
Bibliographic Details
Published inJournal of statistics and data science education pp. 1 - 13
Main Authors Jones-Todd, Charlotte M., Renelle, Amy
Format Journal Article
LanguageEnglish
Published Taylor & Francis Group 04.08.2025
Subjects
Peer grading
Peer marking
Peer review
Programming
Statistics
Online AccessGet full text

Cover

Abstract We investigate the level of agreement between tutor and peer grading of an introductory R programming assessment. Comparing peer and tutor grades we find a strong correlation of 0.848, 95%CI = (0.809, 0.880). Using standard multivariate data analysis techniques we find that tutors and peers grade similarly given a prescriptive criterion. However, when given a subjective criterion, tutors and peers use different schemas to grade. We find that tutors grade the subjective criterion autonomously from the other rubric criteria, whereas peers grade in line with the prescriptive criteria that evaluates the components and structure of the code. In addition, we estimate between-assessor and between-submission variation using a discrete-Beta mixed model and show that between-submission is greater than grader submission for both peers and tutors. Finally, we advocate for the use of peer assessment as a learning exercise and encourage readers to adapt the activity accordingly.
AbstractList We investigate the level of agreement between tutor and peer grading of an introductory R programming assessment. Comparing peer and tutor grades we find a strong correlation of 0.848, 95%CI = (0.809, 0.880). Using standard multivariate data analysis techniques we find that tutors and peers grade similarly given a prescriptive criterion. However, when given a subjective criterion, tutors and peers use different schemas to grade. We find that tutors grade the subjective criterion autonomously from the other rubric criteria, whereas peers grade in line with the prescriptive criteria that evaluates the components and structure of the code. In addition, we estimate between-assessor and between-submission variation using a discrete-Beta mixed model and show that between-submission is greater than grader submission for both peers and tutors. Finally, we advocate for the use of peer assessment as a learning exercise and encourage readers to adapt the activity accordingly.
Author Renelle, Amy
Jones-Todd, Charlotte M.
Author_xml – sequence: 1
  givenname: Charlotte M.
  orcidid: 0000-0003-1201-2781
  surname: Jones-Todd
  fullname: Jones-Todd, Charlotte M.
– sequence: 2
  givenname: Amy
  orcidid: 0000-0002-8952-9239
  surname: Renelle
  fullname: Renelle, Amy
BookMark eNpNkM1OwzAQhC1UJAr0EZDyAin-iZ34WFVQKlWAUDlbm7VTpWrtyk4PfXuStiD2MquZ3Tl892Tkg3eEPDE6ZbSiz1xpoZnSU065nHLZC5U3ZDz4-RCM_u13ZJLSltL-lAlRijF5n2XzsD9AbFPwWWiyT-dinoG32frYhZgvItjWb4YIfLb0XQz2iH1yyr7613M2S8mltHe-eyS3DeySm1z1gXy_vqznb_nqY7Gcz1Y5csFkbhkKRFEjlKBZCUVhpVJIi6oRlJYOuWK8UQg1ZwiVrTQVdVOgVJaVtbTigSwvvTbA1hxiu4d4MgFaczZC3BiIXYs7Z5q6LpBaVmvAQlUAvFQOdN2PVsiqvkteujCGlKJr_voYNQNj88vYDIzNlbH4AW6hcFg
Cites_doi 10.1187/cbe.21-05-0133
10.21105/joss.01686
10.3102/00346543068003249
10.1080/0260293022000001337
10.1016/j.ece.2019.08.002
10.1016/j.jslw.2008.06.002
10.32614/CRAN.package.RTMB
10.1198/tast.2010.09132
10.32614/CRAN.package.palmerpenguins
10.1080/03075079412331382153
10.1080/26939169.2022.2089410
10.1080/08993408.2012.728041
10.1080/00031305.2017.1397549
10.1080/00031305.2015.1093029
10.1007/978-3-319-24277-4
10.1080/02602938.2019.1697424
10.1080/10691898.2020.1870416
10.1080/10691898.2002.11910662
10.1080/10691898.2020.1844106
10.11120/plan.2001.00030020
10.1007/978-3-031-26010-0_12
10.1007/s10648-020-09561-x
10.3102/00346543070003287
10.1080/02602938.2018.1463354
10.1080/02602938.2020.1863911
10.1080/02602938.2019.1643449
ContentType Journal Article
DBID AAYXX
CITATION
DOA
DOI 10.1080/26939169.2025.2520205
DatabaseName CrossRef
DOAJ Directory of Open Access Journals
DatabaseTitle CrossRef
DatabaseTitleList
Database_xml – sequence: 1
  dbid: DOA
  name: DOAJ Open Access Full Text
  url: https://www.doaj.org/
  sourceTypes: Open Website
DeliveryMethod fulltext_linktorsrc
Discipline Mathematics
Statistics
EISSN 2693-9169
EndPage 13
ExternalDocumentID oai_doaj_org_article_fbb4c0d1b9ac468aa276ea9bbbb96c18
10_1080_26939169_2025_2520205
GroupedDBID 8G5
AAFWJ
AAYXX
ABUWG
AFKRA
AFPKN
ALMA_UNASSIGNED_HOLDINGS
AZQEC
BENPR
CCPQU
CITATION
DWQXO
EBS
GNUQQ
GROUPED_DOAJ
GUQSH
H13
M2O
M~E
PHGZM
PHGZT
PQEDU
TDBHL
PUEGO
ID FETCH-LOGICAL-c2315-d1c3cc3bca7a917a44d566c048f3007ec2612f6cab21ca8d8903bf4c56d17b5d3
IEDL.DBID DOA
ISSN 2693-9169
IngestDate Wed Aug 27 01:32:04 EDT 2025
Wed Aug 06 19:15:54 EDT 2025
IsDoiOpenAccess true
IsOpenAccess true
IsPeerReviewed true
IsScholarly true
Language English
License http://creativecommons.org/licenses/by-nc/4.0
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c2315-d1c3cc3bca7a917a44d566c048f3007ec2612f6cab21ca8d8903bf4c56d17b5d3
ORCID 0000-0003-1201-2781
0000-0002-8952-9239
OpenAccessLink https://doaj.org/article/fbb4c0d1b9ac468aa276ea9bbbb96c18
PageCount 13
ParticipantIDs doaj_primary_oai_doaj_org_article_fbb4c0d1b9ac468aa276ea9bbbb96c18
crossref_primary_10_1080_26939169_2025_2520205
PublicationCentury 2000
PublicationDate 2025-08-04
PublicationDateYYYYMMDD 2025-08-04
PublicationDate_xml – month: 08
  year: 2025
  text: 2025-08-04
  day: 04
PublicationDecade 2020
PublicationTitle Journal of statistics and data science education
PublicationYear 2025
Publisher Taylor & Francis Group
Publisher_xml – name: Taylor & Francis Group
References e_1_3_4_4_1
e_1_3_4_3_1
Medeiros R. (e_1_3_4_18_1) 2023
e_1_3_4_9_1
e_1_3_4_8_1
e_1_3_4_7_1
e_1_3_4_20_1
e_1_3_4_6_1
e_1_3_4_5_1
e_1_3_4_23_1
e_1_3_4_24_1
e_1_3_4_21_1
e_1_3_4_27_1
e_1_3_4_28_1
e_1_3_4_26_1
e_1_3_4_29_1
Smaill C. (e_1_3_4_25_1) 2011
e_1_3_4_31_1
CANVAS (e_1_3_4_2_1) 2022
e_1_3_4_30_1
e_1_3_4_12_1
e_1_3_4_35_1
e_1_3_4_13_1
e_1_3_4_34_1
e_1_3_4_10_1
e_1_3_4_33_1
e_1_3_4_11_1
e_1_3_4_32_1
e_1_3_4_16_1
e_1_3_4_17_1
e_1_3_4_14_1
e_1_3_4_15_1
e_1_3_4_19_1
R Core Team (e_1_3_4_22_1) 2022
References_xml – ident: e_1_3_4_8_1
  doi: 10.1187/cbe.21-05-0133
– ident: e_1_3_4_33_1
– ident: e_1_3_4_19_1
– ident: e_1_3_4_35_1
  doi: 10.21105/joss.01686
– ident: e_1_3_4_30_1
  doi: 10.3102/00346543068003249
– ident: e_1_3_4_21_1
  doi: 10.1080/0260293022000001337
– ident: e_1_3_4_24_1
  doi: 10.1016/j.ece.2019.08.002
– ident: e_1_3_4_17_1
  doi: 10.1016/j.jslw.2008.06.002
– ident: e_1_3_4_16_1
  doi: 10.32614/CRAN.package.RTMB
– ident: e_1_3_4_20_1
  doi: 10.1198/tast.2010.09132
– ident: e_1_3_4_12_1
  doi: 10.32614/CRAN.package.palmerpenguins
– ident: e_1_3_4_28_1
  doi: 10.1080/03075079412331382153
– ident: e_1_3_4_32_1
– ident: e_1_3_4_31_1
  doi: 10.1080/26939169.2022.2089410
– ident: e_1_3_4_27_1
– ident: e_1_3_4_29_1
  doi: 10.1080/08993408.2012.728041
– ident: e_1_3_4_4_1
  doi: 10.1080/00031305.2017.1397549
– volume-title: Instructure
  year: 2022
  ident: e_1_3_4_2_1
– ident: e_1_3_4_5_1
  doi: 10.1080/00031305.2015.1093029
– ident: e_1_3_4_34_1
  doi: 10.1007/978-3-319-24277-4
– ident: e_1_3_4_10_1
  doi: 10.1080/02602938.2019.1697424
– ident: e_1_3_4_13_1
  doi: 10.1080/10691898.2020.1870416
– ident: e_1_3_4_23_1
  doi: 10.1080/10691898.2002.11910662
– ident: e_1_3_4_26_1
  doi: 10.1080/10691898.2020.1844106
– ident: e_1_3_4_14_1
  doi: 10.11120/plan.2001.00030020
– start-page: 22
  volume-title: 2011 ASEE Annual Conference & Exposition,
  year: 2011
  ident: e_1_3_4_25_1
– ident: e_1_3_4_11_1
– volume-title: R : A Language and Environment for Statistical Computing
  year: 2022
  ident: e_1_3_4_22_1
– start-page: 167
  volume-title: Teaching Biostatistics in Medicine and Allied Health Sciences
  year: 2023
  ident: e_1_3_4_18_1
  doi: 10.1007/978-3-031-26010-0_12
– ident: e_1_3_4_9_1
  doi: 10.1007/s10648-020-09561-x
– ident: e_1_3_4_6_1
  doi: 10.3102/00346543070003287
– ident: e_1_3_4_3_1
  doi: 10.1080/02602938.2018.1463354
– ident: e_1_3_4_7_1
  doi: 10.1080/02602938.2020.1863911
– ident: e_1_3_4_15_1
  doi: 10.1080/02602938.2019.1643449
SSID ssj0002513373
Score 2.2991853
Snippet We investigate the level of agreement between tutor and peer grading of an introductory R programming assessment. Comparing peer and tutor grades we find a...
SourceID doaj
crossref
SourceType Open Website
Index Database
StartPage 1
SubjectTerms Peer grading
Peer marking
Peer review
Programming
Statistics
Title A Comparison of Peer- and Tutor-Grading of an Introductory R Coding Assessment
URI https://doaj.org/article/fbb4c0d1b9ac468aa276ea9bbbb96c18
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV09T8MwELVQpzIgKCDKlzywuo1jJ47HUlEKUiuEWqlb5M8xRaUMLPx2fE4aZWPBQyLFSWS9i3N30t17CD04kzChFSNSME24LxTR3Ekig_PTYaM7HaUTFst8vuavm2zTkfqCmrCaHrgGbuy15iaxVEtleF4olYrcKanDkLmhsc03-LxOMgX_4BRkSwQ7tOwUyTjNJfSYQm9Kmo3SLJxAsq7jjDqc_dG5zE7RSRMV4km9mjN05KoBOl60lKqfA9SHsLBmVT5HywmetgKCeOvxm3M7glVl8QpICcjzLtbGw5Sq8AuUowOx63b3jd_Do3Fu0pJyXqD17Gk1nZNGGYGYEI9lxFLDjGHaKKFCvqU4tyEsM2E3ehacvjNADOZzo3RKjSpsIROmPTdZbqnQmWWXqFdtK3eFsHA0ZDSOU2MlD0NRWYAH8x6IcVIxRKMDROVHTYBR0oZX9IBpCZiWDaZD9AhAtjcDf3W8EKxaNlYt_7Lq9X-85Ab1YWGxXI_fot5-9-XuQgix1_fxawnHxc_TL_V9w8Y
linkProvider Directory of Open Access Journals
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+Comparison+of+Peer-+and+Tutor-Grading+of+an+Introductory+R+Coding+Assessment&rft.jtitle=Journal+of+statistics+and+data+science+education&rft.au=Charlotte+M.+Jones-Todd&rft.au=Amy+Renelle&rft.date=2025-08-04&rft.pub=Taylor+%26+Francis+Group&rft.eissn=2693-9169&rft.spage=1&rft.epage=13&rft_id=info:doi/10.1080%2F26939169.2025.2520205&rft.externalDBID=DOA&rft.externalDocID=oai_doaj_org_article_fbb4c0d1b9ac468aa276ea9bbbb96c18
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2693-9169&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2693-9169&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2693-9169&client=summon