Design of an Active Learning System with Human Correction for Content Analysis

Nancy McCracken, Jasy Suet Yan Liew, Kevin Crowston
2014 Proceedings of the Workshop on Interactive Language Learning, Visualization, and Interfaces   unpublished
Our research investigation focuses on the role of humans in supplying corrected examples in active learning cycles, an important aspect of deploying active learning in practice. In this paper, we discuss sampling strategies and sampling sizes in setting up an active learning system for human experiments in the task of content analysis, which involves labeling concepts in large volumes of text. The cost of conducting comprehensive human subject studies to experimentally determine the effects of
more » ... ine the effects of sampling sizes and sampling sizes is high. To reduce those costs, we first applied an active learning simulation approach to test the effect of different sampling strategies and sampling sizes on machine learning (ML) performance in order to select a smaller set of parameters to be evaluated in human subject studies.
doi:10.3115/v1/w14-3109 fatcat:7vaflxktfrbqbkfqmwls4vd7xm