Journal

Ontology-Based Generation of Medical, Multi-term MCQs

Publication Information

Authors:

  • Jared Leo, The University of Manchester
  • Ghader Kurdi, The University of Manchester
  • N. Matentzoglu, The University of Manchester
  • Bijan Parsia, The University of Manchester
  • Uli Sattler, The University of Manchester
  • S. Forge, Elsevier
  • G. Donato, Elsevier
  • W. Dowling, Elsevier

Pages:

  • 145-188

Keywords:

  • OWL, Ontology, Medical education, Student assessment, e-learning, Question generation

Abstract:

  • Designing good multiple choice questions (MCQs) for education and assessment is time consuming and error-prone. An abundance of structured and semi-structured data has led to the development of automatic MCQ generation methods. Recently, ontologies have emerged as powerful tools to enable the automatic generation of MCQs. However, current question generation approaches focus on knowledge recall questions. In addition, questions that have so far been generated are, compared to manually created ones, simple and cover only a small subset of the required question complexity space in the education and assessment domain. In this paper, we focus on addressing the limitations of previous approaches by generating questions with complex stems that are suitable for scenarios beyond mere knowledge recall. We present a novel ontology-based approach that exploits classes and existential restrictions to generate case-based questions. Our contribution lies in: (1) the specification of procedure for generating case-based questions which involve (a) assembling complex stems, (b) selecting suitable options, and (c) providing explanations for option correctness/incorrectness, (2) an implementation of the procedure using a medical ontology and (3) and evaluation of our generation technique to test question quality and their suitability in practise. We implement our approach as an application for a medical education scenario on top of a large knowledge base in the medical domain. We generate more than 3 million questions for four physician specialities and evaluate our approach in a user study with 15 medical experts. We find that using a stratified random sample of 435 questions out of which 316 were rated by two experts, 129 (30%) are considered appropriate to be used in exams by both experts and a further 216 (50%) by at least one expert.