A SMART VALIDATION ARCHITECTURE FOR COMPUTER-BASED EXAMINATION

ABSTRACT

Computer-based tests are now one of the prominent methods of administering examinations of various kinds due to their preciseness, predictability and ease of grading especially when multiple-choice methods are used. Despite these motivations, multiple choice-based systems are often fraught with problems such as repeated questions, repeated options for responses in certain cases, questions containing missing diagrams etc., which constitutes a research gap. In this paper, a smart validation architecture is proposed to address these limitations. The architecture is a three-phased methodology consisting of an Upload Engine, a Smart Assessor and a Generate CBT Engine. The Upload engine accepts the test questions in a certain specified format which is then transferred to the SA. The SA is a validation module which accepts assessment questions and invokes the appropriate error-checking algorithm to ensure the correctness of the question. This is achieved with extensible libraries of algorithms developed for each peculiar problem to ensure only unambiguous questions are delivered to the next phase. The Generate CBT Engine is an artificial intelligence system for automatically creating new examination questions without human effort. The proposed scheme uses Python Programming language for its implementation. The evaluation process consists of validity experiments and discriminative performance metrics on the AI-generated module. The experimental data showed a significant prevalence of errors in most CBT systems thereby indicating the need for the proposed design. The discriminative evaluations were based on the system’s precision, recall and FI-Score of the AI module. Findings indicate that the proposed scheme presents a promising research direction for improving the usability and reliability of the CBT framework with a less stressful examination experience for both candidates and proctors/invigilators.

Keywords: Computer Based Test, Educational assessment, Smart Applications, Test performance, Validation