An Interview Reflection on “Intelligent Tutoring Goes to School in the Big City”

In IJAIED 26 (1)

Publication information

13-24

Cognitive model, Cognitive Tutors, intelligent tutroing systems, Learning Science, School-based field studies

Abstract

Our 1997 article in IJAIED reported on a study that showed that a new algebra curriculum with an embedded intelligent tutoring system (the Algebra Cognitive Tutor) dramatically enhanced high-school students’ learning. The main motivation for the study was to demonstrate that intelligent tutors that have cognitive science research embedded in them could have real impact in schools. This study was one of the first large-scale classroom evaluations of the integrated use of an Intelligent Tutoring System (ITS) in high schools. A core challenge was figuring out how to embed this new technology into a curriculum and into the existing social context of schools. A key element of the study design was to include multiple kinds of assessments, including standardized test items and items measuring complex problem solving and use of representations. The results were powerful: “On average the 470 students in experimental classes outperformed students in comparison classes by 15 % on standardized tests and 100 % on tests targeting the [course] objectives.” We suggested that the study was evidence “that laboratory tutoring systems can be scaled up and made to work, both technically and pedagogically, in real and unforgiving settings like urban high schools.” Since this study, many more classroom studies comparing instruction that includes an ITS against business as usual have been conducted, often showing advantages for the ITS-enhanced curricula. More rigorous randomized field trials are now more commonplace, but the approach of using multiple assessments in large-scale randomized field trials has not caught on. Cognitive task analysis will remain fundamental to the success of ITSs. A key remaining question for ITS is to find out how they can be used most effectively to support open-ended problem solving, either online or offline. Given all the recent excitement around Massive Open Online Courses (MOOCs), it is interesting to note that our field of Artificial Intelligence in Education has been making huge, less recognized, progress with impact on millions of students and with the majority of those students finishing the course!