Publication Information

Authors:

• Michelle Barrett, ACT, Inc.
• Bingnan Jiang, ACT, Inc.
• Bridget E. Feagler, ACT,Inc.

Pages:

• 28-47

Keywords:

• Computerized adaptive testing, Smart authoring, Assessment, Shadow test, Automated test assembly

Abstract:

• The appeal of a shorter testing time makes a computer adaptive testing approach highly desirable for use in multiple assessment and learning contexts. However, for those who have been tasked with designing, configuring, and deploying adaptive tests for operational use at scale, preparing an adaptive test is anything but simple. The process often involves a complex interplay among psychometricians, content experts, and technologists who operate with different vocabularies and subject matter expertise. This paper presents the authors’ experience of developing smart platforms for designing, configuring, and deploying adaptive assessments. We present six design principles for the development of such systems. Through an example of an authoring system now used in production, we discuss smart feedback loops built into the system and how they support efficient iteration. We conclude that it is not possible to overstate the importance of process transition support when launching smart platforms, and that a thoughtful and integrated user experience that allows each of the people in the process to work in a context most accessible to them given their background and experience yields substantial business value.