How can you tell whether a study aptitude test is good? Psychologists have detailed checklists with hundreds of questions—but here’s a much shorter version with 5 key questions every admissions professional should ask.
The wrong question to start with: “Who else is already using this test?”
It’s tempting to assume that if other institutions use a test, it must be good. But that’s not always the case. Some tests that don’t meet even basic scientific standards are still widely used, especially in the HR sector.
Ask these 5 questions instead:
1. Who developed the test?
Test development should be led by experts in psychological diagnostics (psychometricians). Would you let someone without medical degree and experience perform heart surgery?
2. How was the test developed?
The test provider should describe the development process and refer to established scientific standards—e.g., requirement analyses and empirical evaluation.
3. What’s the reliability?
You should expect reliability coefficients of .80 or higher, some tests even reach .90+. Short screening tools may be acceptable around .70. You should also aks which reliability measure was used (Cronbach’s alpha, retest,…).
4. What’s the validity?
Some test providers offer no information on validity at all. While conducting predictive validity studies is challenging, especially in line with data protection rules, responsible providers will take this seriously.
Look for:
- Validity studies (ideally published with peer-review)
- Correlations with study GPA (often between .30 and .50), ideally in longitudinal studies, or with other external criteria indicating validity
- Description of the methods (raw vs. corrected correlations, regression models, etc.)
5. How is data protection handled?
This can be hard to judge without being a legal expert, but it’s critical—especially for universities in Europe. Some non-EU test providers transfer candidate data to third parties or require consent to terms that don’t comply with GDPR. That’s a serious risk for your institution.
