Game theory based peer grading mechanisms for MOOCs

William Wu, Christos Tzamos, Constantinos Daskalakis, Matthew Weinberg, Nicolaas Kaashoek

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Scopus citations

Abstract

An efficient peer grading mechanism is proposed for grading the multitude of assignments in online courses. This novel approach is based on game theory and mechanism design. A set of assumptions and a mathematical model is ratified to simulate the dominant strategy behavior of students in a given mechanism. A benchmark function accounting for grade accuracy and workload is established to quantitatively compare effectiveness and scalability of various mechanisms. After multiple iterations of mechanisms under increasingly realistic assumptions, three are proposed: Calibration, Improved Calibration, and Deduction. The Calibration mechanism performs as predicted by game theory when tested in an online crowd-sourced experiment, but fails when students are assumed to communicate. The Improved Calibration mechanism addresses this assumption, but at the cost of more effort spent grading. The Deduction mechanism performs relatively well in the benchmark, outperforming the Calibration, Improved Calibration, traditional automated, and traditional peer grading systems. The mathematical model and benchmark opens the way for future derivative works to be performed and compared.

Original languageEnglish (US)
Title of host publicationL@S 2015 - 2nd ACM Conference on Learning at Scale
PublisherAssociation for Computing Machinery, Inc
Pages281-286
Number of pages6
ISBN (Electronic)9781450334112
DOIs
StatePublished - Mar 14 2015
Externally publishedYes
Event2nd ACM Conference on Learning at Scale, L@S 2015 - Vancouver, Canada
Duration: Mar 14 2015Mar 18 2015

Publication series

NameL@S 2015 - 2nd ACM Conference on Learning at Scale

Other

Other2nd ACM Conference on Learning at Scale, L@S 2015
CountryCanada
CityVancouver
Period3/14/153/18/15

All Science Journal Classification (ASJC) codes

  • Software
  • Education
  • Computer Science Applications
  • Computer Networks and Communications

Keywords

  • Game theory
  • Learning at scale
  • MOOC
  • Massive open online courses
  • Mechanism design
  • Peer grading

Fingerprint Dive into the research topics of 'Game theory based peer grading mechanisms for MOOCs'. Together they form a unique fingerprint.

Cite this