Automated Test Assembly for Handling Learner Cold-Start in Large-Scale Assessments

Publication Information

Authors:

  • Jill-Jênn Vie, RIKEN Center for Advanced Intelligence Project
  • Fabrice Papineau,
  • Éric Bruillard, École normale supérieure de Paris-Saclay - STEF
  • Yolaine Bourda,

Pages:

  • 616-631

Keywords:

  • Cold-start, Test-size reduction, Learning analytics, Determinantal point processes, Multistage testing, Cognitive diagnosis

Abstract:

  • In large-scale assessments such as the ones encountered in MOOCs, a lot of usage data is available because of the number of learners involved. Newcomers, that just arrive on a MOOC, have various backgrounds in terms of knowledge, but the platform hardly knows anything about them. Therefore, it is crucial to elicit their knowledge fast, in order to personalize their learning experience. Such a problem has been called learner cold-start. We present in this article an algorithm for sampling a group of initial, diverse questions for a newcomer, based on a method recently used in machine learning: determinantal point processes. We show, using real data, that our method outperforms existing techniques such as uncertainty sampling, and can provide useful feedback to the learner over their strong and weak points.