Applying human computation methods to information science
___________________________________ Gautam Pant ii To my wife, Shuhong and my children, Brady, Ronan and Miliani iii ACKNOWLEDGMENTS I wish to thank my thesis supervisor Padmini Srinivasan for her guidance and vision as I developed and executed this research the emerging field. In particular, I appreciate her direction, vision and ideas on how human computation can contribute to a new paradigm for accomplishing tasks in many different disciplines, including Information Science. I also wish to
... e. I also wish to thank the others on my thesis committee for their direction on how to proceed in my research during the most challenging times, their prudent guidance, and their careful feedback on my ideas in human computation. I also wish to thank those students and faculty in the University of Iowa Text Retrieval group, many of whom have contributed in different ways to support this research, including providing useful feedback on new ideas, sharing new perspectives on long-standing problems in Information Retrieval, assisting me with research design, and reviewing research papers. Last but certainly not least, I wish to thank my wife and children who have patiently encouraged me along the way. iv ABSTRACT Human Computation methods such as crowdsourcing and games with a purpose (GWAP) have each recently drawn considerable attention for their ability to synergize the strengths of people and technology to accomplish tasks that are challenging for either to do well alone. Despite this increased attention, much of this transformation has been focused on a few selected areas of information science. This thesis contributes to the field of human computation as it applies to areas of information science, particularly information retrieval (IR). We begin by discussing the merits and limitations of applying crowdsourcing and game-based approaches to information science. We then develop a framework that examines the value of using crowdsourcing and game mechanisms to each step of an IR model. We identify three areas of the IR model that our framework indicates are likely to benefit from the application of human computation methods: acronym identification and resolution, relevance assessment, and query formulation. We conduct experiments that employ human computation methods, evaluate the benefits of these methods and report our findings. We conclude that employing human computation methods such as crowdsourcing and games, can improve the accuracy of many tasks currently being done by machine methods alone. We demonstrate that the best results can be achieved when human computation methods augment computer-based IR processes, providing an extra level of skills, abilities, and knowledge that computers cannot easily replicate.