We have recently implemented “adaptive testing” for an admission test of a renowned Business School. This approach dynamically adjusts question difficulty based on individual performance, offering both advantages and considerations worth exploring.
What Is Adaptive Testing?
Adaptive testing tailors the difficulty of questions to demonstrated abilities, meaning that stronger participants get more difficult and weaker participants more easy tasks.
Advantages of Adaptive Testing
- Efficiency: Shorter tests can be achieved without compromising accuracy due to tailored question difficulty.
- Security: Varied question sets provide less opportunities for cheating (e.g. by hidden communication between participants).
- Sustainability: Once established, adaptive tests require less maintenance compared to linear tests.
Challenges to Consider
- User Experience: Some participants may find the format less intuitive, as they cannot navigate at will between all tasks of a module. In addition, strong participants will find the test more stressful.
- Initial Investment: Setting up adaptive testing demands significant resources, including IT development, item pool creation, statistical modeling and a lot of pre-testing.
- Transparency: Grading transparency may be compromised as raw scores do not perfectly correspond to test results, making it more difficult for participants to understand how their test scores are calculated.
Our Adaptive Testing Model
Our adaptive test consists of two modules assessing quantitative skills and data literacy (e.g. analysis of diagrams and tables), with 32 tasks in 70 minutes. Tasks are dynamically adjusted for each module based on an initial performance assessment after 8 randomly chosen items with medium difficulty, aiming to provide a fair and accurate evaluation.
Our Journey Thus Far
While the implementation of adaptive testing has been met with positive reception from participants and the Business School, we acknowledge the rigorous development process it entails. Technical challenges and meticulous planning were necessary steps in ensuring a smooth transition.
