BLAh: Boolean Logic Analysis for Graded Student Response Data

Andrew S. Lan, Andrew E. Waters, Christoph Studer, Richard G. Baraniuk

Research output: Contribution to journalArticle

2 Scopus citations

Abstract

Machine learning (ML) models and algorithms can enable a personalized learning experience for students in an inexpensive and scalable manner. At the heart of ML-driven personalized learning is the automated analysis of student responses to assessment items. Existing statistical models for this task enable the estimation of student knowledge and question difficulty solely from graded response data with only minimal effort from instructors. However, most existing student-response models are generalized linear models, meaning that they characterize the probability that a student answers a question correctly through a linear combination of their knowledge and the question's difficulty with respect to each concept that is being assessed. Such models cannot characterize complicated, nonlinear student-response associations and, hence, lack human interpretability in practice. In this paper, we propose a nonlinear student-response model called Boolean logic analysis (BLAh) that models a student's binary-valued graded response to a question as the output of a Boolean logic function. We develop a Markov chain Monte Carlo inference algorithm that learns the Boolean logic functions for each question solely from graded response data. A refined BLAh model improves the identifiability, tractability, and interpretability by considering a restricted set of ordered Boolean logic functions. Experimental results on a variety of real-world educational datasets demonstrate that BLAh not only achieves best-in-class prediction performance on unobserved student responses on some datasets but also provides easily interpretable parameters when questions are tagged with metadata by domain experts, which can provide useful feedback to instructors and content designers to improve the quality of assessment items.

Original languageEnglish (US)
Article number7967714
Pages (from-to)754-764
Number of pages11
JournalIEEE Journal on Selected Topics in Signal Processing
Volume11
Issue number5
DOIs
StatePublished - Aug 2017

Keywords

  • Boolean logic
  • Markov Chain Monte Carlo
  • machine learning
  • personalized learning
  • student-response data

ASJC Scopus subject areas

  • Signal Processing
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'BLAh: Boolean Logic Analysis for Graded Student Response Data'. Together they form a unique fingerprint.

Cite this