The regulations accompanying the pandemic make it necessary in part for certain university examinations and selection procedures for students to be conducted by means of proctoring. With proctoring, examinees take their test at home on their own computer and are supervised digitally by so-called proctors.
But not all proctoring is the same! There are different ways in which digital proctoring can be implemented. In the following, we briefly present our solution.
AI-based proctoring vs. live supervision
In our view, proctoring variants in which an “artificial intelligence” analyses all the participants’ movements and reports attempts to deceive or suspicious behaviour on the basis of the smallest deviations in behaviour are rightly criticised. The examinees are not only monitored via the webcam and are not allowed to avert their gaze from the screen or even move their head, for example, but all background noises and the permanent contact of the hands with the keyboard are also recorded. The disadvantages for the acceptance of the entire test procedure are obvious.
For us, human assessment of behaviour is therefore essential, also because AI cannot (yet) assess behaviour reliably enough. Normal human behaviour, such as looking up when working at a computer for a long time, leaning back to think or even mumbling quietly to oneself, should never be associated with the risk of exclusion from an exam.
How do we implement digital supervision?
With our approach, we have tried to simulate the “analogue” supervision during a test or examination on site. We rely on live supervision via webcam and screen transmission by trained invigilators, the so-called proctors. The same rules apply for proctoring as for a face-to-face examination: the participants are allowed to take notes with pen and paper, to eat and drink during the examination, and to get up and leave the room (e.g. to go to the toilet).
The proctors can intervene in the event of possible attempts at deception and, for example, have the tools (pen, paper), the workplace or the entire room shown to them. It is important to us that the latter in particular only happens if there are reasonable grounds for suspicion. On the test days, at least two ITB counsellors take over the project management, i.e. they serve as contact persons for the proctors and coordinate the entire process. Both the webcam images and the screen monitoring are additionally recorded. However, this recording is only used for subsequent testing, for example if participants report malfunctions due to technical problems, and is deleted immediately afterwards. We take data protection and security very seriously in proctoring tests.
And we attach great importance to ensuring that the participants can prepare thoroughly for the test. They go through a comprehensive system check with their end devices in advance, and we have also prepared all the important information for participants on our website.
A first study (Stegt & Hofmann, 2020) comparing the results of the TM-WISO in a face-to-face implementation in test centres with an implementation with proctoring showed no significant differences. This confirms our approach of using proctoring to come as close as possible to analogue supervision on site. Currently, we are also conducting acceptance surveys. Initial results suggest that the procedures are just as well accepted by participants as on-site testing.
- Stegt, S. J. & Hofmann, A. (2020). Eignungstests in der Coronapandemie: Fachspezifische Onlinetests mit Proctoring zur Auswahl von Bachelor- und Masterstudierenden. Qualität in der Wissenschaft, 14(3), 84-90 (german).
- Our information page on proctoring tests
- Information for participants