Mathematical language processing: Automatic grading and feedback for open response mathematical questions

Andrew S. Lan, Divyanshu Vats, Andrew E. Waters, Richard G. Baraniuk

Research output: Chapter in Book/Report/Conference proceedingConference contribution

20 Scopus citations

Abstract

While computer and communication technologies have provided effective means to scale up many aspects of education, the submission and grading of assessments such as homework assignments and tests remains a weak link. In this paper, we study the problem of automatically grading the kinds of open response mathematical questions that figure prominently in STEM (science, technology, engineering, and mathematics) courses. Our data-driven framework for mathematical language processing (MLP) leverages solution data from a large number of learners to evaluate the correctness of their solutions, assign partial-credit scores, and provide feedback to each learner on the likely locations of any errors. MLP takes inspiration from the success of natural language processing for text data and comprises three main steps. First, we convert each solution to an open response mathematical question into a series of numerical features. Second, we cluster the features from several solutions to uncover the structures of correct, partially correct, and incorrect solutions. We develop two different clustering approaches, one that leverages generic clustering algorithms and one based on Bayesian nonparametrics. Third, we automatically grade the remaining (potentially large number of) solutions based on their assigned cluster and one instructor-provided grade per cluster. As a bonus, we can track the cluster assignment of each step of a multistep solution and determine when it departs from a cluster of correct solutions, which enables us to indicate the likely locations of errors to learners. We test and validate MLP on real-world MOOC data to demonstrate how it can substantially reduce the human effort required in large-scale educational platforms.

Original languageEnglish (US)
Title of host publicationL@S 2015 - 2nd ACM Conference on Learning at Scale
PublisherAssociation for Computing Machinery, Inc
Pages167-176
Number of pages10
ISBN (Electronic)9781450334112
DOIs
StatePublished - Jan 1 2015
Event2nd ACM Conference on Learning at Scale, L@S 2015 - Vancouver, Canada
Duration: Mar 14 2015Mar 18 2015

Other

Other2nd ACM Conference on Learning at Scale, L@S 2015
CountryCanada
CityVancouver
Period3/14/153/18/15

Keywords

  • Assessment
  • Automatic grading
  • Bayesian nonparametrics
  • Clustering
  • Feedback
  • Machine learning
  • Mathematical language processing

ASJC Scopus subject areas

  • Software
  • Education
  • Computer Science Applications
  • Computer Networks and Communications

Fingerprint Dive into the research topics of 'Mathematical language processing: Automatic grading and feedback for open response mathematical questions'. Together they form a unique fingerprint.

Cite this