Evaluation Methodologies [chapter]

Ivana Chingovska, André Anjos, Sébastien Marcel
2014 Advances in Computer Vision and Pattern Recognition  
We believe that every effectiveness evaluation should be replicated at least in order to verify the original results and to indicate evaluated e-learning system's advantages or disadvantages. This paper presents the methodology for conducting controlled experiment replication, as well as, results of a controlled experiment and an internal replication that investigated the effectiveness of intelligent authoring shell eXtended Tutor-Expert System (xTEx-Sys). The initial and the replicated
more » ... nt were based on our approach that combines classical two-group experimental design and with factoral design. A trait that distinguishes this approach from others is the existence of arbitrary number of checkpoint-tests to determine the effectiveness in intermediate states. We call it a pre-and-post test control group experimental design with checkpoint-tests. The gained results revealed small or even negative effect sizes, which could be explained by the fact that the xTEx-Sys's domain knowledge presentation is rather novel for students and therefore difficult to grasp and apply in earlier phases of the experiment. In order to develop and improve the xTEx-Sys, further experiments must be conducted.
doi:10.1007/978-1-4471-6524-8_10 fatcat:zmat3qw5nvgzrbrd3lkpk35q2u