BayesRank: A bayesian approach to ranked peer grading

Andrew E. Waters, David Tinapple, Richard G. Baraniuk

Research output: Chapter in Book/Report/Conference proceedingConference contribution

22 Scopus citations


Advances in online and computer supported education afford exciting opportunities to revolutionize the classroom, while also presenting a number of new challenges not faced in traditional educational settings. Foremost among these challenges is the problem of accurately and efficiently evaluating learner work as the class size grows, which is directly related to the larger goal of providing quality, timely, and actionable formative feedback. Recently there has been a surge in interest in using peer grading methods coupled with machine learning to accurately and fairly evaluate learner work while alleviating the instructor bottleneck and grading overload. Prior work in peer grading almost exclusively focuses on numerically scored grades - either real-valued or ordinal. In this work, we consider the implications of peer ranking in which learners rank a small subset of peer work from strongest to weakest, and propose new types of computational analyses that can be applied to this ranking data. We adopt a Bayesian approach to the ranked peer grading problem and develop a novel model and method for utilizing ranked peer-grading data. We additionally develop a novel procedure for adaptively identifying which work should be ranked by particular peers in order to dynamically resolve ambiguity in the data and rapidly resolve a clearer picture of learner performance. We showcase our results on both synthetic and several real-world educational datasets.

Original languageEnglish (US)
Title of host publicationL@S 2015 - 2nd ACM Conference on Learning at Scale
PublisherAssociation for Computing Machinery, Inc
Number of pages7
ISBN (Print)9781450334112
StatePublished - Mar 14 2015
Event2nd ACM Conference on Learning at Scale, L@S 2015 - Vancouver, Canada
Duration: Mar 14 2015Mar 18 2015


Other2nd ACM Conference on Learning at Scale, L@S 2015

ASJC Scopus subject areas

  • Software
  • Education
  • Computer Science Applications
  • Computer Networks and Communications


Dive into the research topics of 'BayesRank: A bayesian approach to ranked peer grading'. Together they form a unique fingerprint.

Cite this