Game theory based peer grading mechanisms for MOOCs

William Wu, Christos Tzamos, Constantinos Daskalakis, Matthew Weinberg, Nicolaas Kaashoek

Research output: Chapter in Book/Report/Conference proceedingConference contribution

12 Scopus citations


An efficient peer grading mechanism is proposed for grading the multitude of assignments in online courses. This novel approach is based on game theory and mechanism design. A set of assumptions and a mathematical model is ratified to simulate the dominant strategy behavior of students in a given mechanism. A benchmark function accounting for grade accuracy and workload is established to quantitatively compare effectiveness and scalability of various mechanisms. After multiple iterations of mechanisms under increasingly realistic assumptions, three are proposed: Calibration, Improved Calibration, and Deduction. The Calibration mechanism performs as predicted by game theory when tested in an online crowd-sourced experiment, but fails when students are assumed to communicate. The Improved Calibration mechanism addresses this assumption, but at the cost of more effort spent grading. The Deduction mechanism performs relatively well in the benchmark, outperforming the Calibration, Improved Calibration, traditional automated, and traditional peer grading systems. The mathematical model and benchmark opens the way for future derivative works to be performed and compared.

Original languageEnglish (US)
Title of host publicationL@S 2015 - 2nd ACM Conference on Learning at Scale
PublisherAssociation for Computing Machinery
Number of pages6
ISBN (Electronic)9781450334112
StatePublished - Mar 14 2015
Externally publishedYes
Event2nd ACM Conference on Learning at Scale, L@S 2015 - Vancouver, Canada
Duration: Mar 14 2015Mar 18 2015

Publication series

NameL@S 2015 - 2nd ACM Conference on Learning at Scale


Other2nd ACM Conference on Learning at Scale, L@S 2015

All Science Journal Classification (ASJC) codes

  • Software
  • Education
  • Computer Science Applications
  • Computer Networks and Communications


  • Game theory
  • Learning at scale
  • MOOC
  • Massive open online courses
  • Mechanism design
  • Peer grading


Dive into the research topics of 'Game theory based peer grading mechanisms for MOOCs'. Together they form a unique fingerprint.

Cite this