Evaluation of Computer Adaptive Testing Systems [chapter]

Anastasios A. Economides, Chrysostomos Roupas
Information Security and Ethics  
Many educational organizations are trying to reduce the cost of the exams, the workload and delay of scoring, and the human errors. Also, they try to increase the accuracy and efficiency of the testing. Recently, most examination organizations use Computer Adaptive Testing (CAT) as the method for large scale testing. This paper investigates the current state of CAT systems and identifies their strengths and weaknesses. It evaluates ten CAT systems using an evaluation framework of fifteen
more » ... categorized into three dimensions: Educational, Technical and Economical. The results show that the majority of the CAT systems give priority to security, reliability and maintainability. However, they do not offer to the examinee any advanced support and functionalities. Also, the feedback to the examinee is limited and the presentation of the items is poor. Recommendations are made in order to enhance the overall quality of a CAT system. For example, alternative multimedia items should be available so that the examinee would choose his preferred media type. Feedback could be improved by providing more information to the examinee or providing information anytime the examinee wished. Abstract: Many educational organizations are trying to reduce the cost of the exams, the workload and delay of scoring, and the human errors. Also, they try to increase the accuracy and efficiency of the testing. Recently, most examination organizations use Computer Adaptive Testing (CAT) as the method for large scale testing. This paper investigates the current state of CAT systems and identifies their strengths and weaknesses. It evaluates ten CAT systems using an evaluation framework of fifteen domains categorized into three dimensions: Educational, Technical and Economical. The results show that the majority of the CAT systems give priority to security, reliability and maintainability. However, they do not offer to the examinee any advanced support and functionalities. Also, the feedback to the examinee is limited and the presentation of the items is poor. Recommendations are made in order to enhance the overall quality of a CAT system. For example, alternative multimedia items should be available so that the examinee would choose his preferred media type. Feedback could be improved by providing more information to the examinee or providing information anytime the examinee wished.
doi:10.4018/978-1-59904-937-3.ch046 fatcat:6wfa2dumsrdnzdu3hyyfalwf4e