1,056 Hits in 5.9 sec

The Perils of Using Mechanical Turk to Evaluate Open-Ended Text Generation

Marzena Karpinska, Nader Akoury, Mohit Iyyer
2021 Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing   unpublished
In this paper, we first conduct a survey of 45 open-ended text generation papers and find that the vast majority of them fail to report crucial details about their AMT tasks, hindering reproducibility.  ...  Recent text generation research has increasingly focused on open-ended domains such as story and poetry generation.  ...  Acknowledgments We thank the reviewers for their insightful comments. We would also like to thank the UMass NLP group for the great advice on the draft of this paper.  ... 
doi:10.18653/v1/2021.emnlp-main.97 fatcat:dhjqztoyyrdznidqknpcow3wk4

Law and Psychology Grows Up, Goes Online, and Replicates

Krin Irvine, David A. Hoffman, Tess Wilkinson-Ryan
2018 Journal of Empirical Legal Studies  
Mueller, Using Mechanical Turk to Study Clinical Populations, 1 CLINICAL PSYCHOL.  ...  Amazon Mechanical Turk in Experimental Legal Studies In part because of the relative youth of the discipline, legal scholars have not generally focused on the replicability of their experimental studies  ...  Generally speaking, law and psychology papers focus on main effects, not individual differences. 75 However, especially with a large number of subjects, the effects of such investigations can be productive  ... 
doi:10.1111/jels.12180 fatcat:l5nnldfxjbao3nxxgxbsvkwmki

Chain Reactions

Carrie J. Cai, Shamsi T. Iqbal, Jaime Teevan
2016 Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems - CHI '16  
Through a series of crowd-based studies, we look at how various microtasks can be chained together to improve efficiency and minimize mental demand, focusing on the writing domain.  ...  Microtasks are small units of work designed to be completed individually, eventually contributing to a larger goal.  ...  We are also grateful to Rob Miller, William Li, and Ramesh Sridharan for their feedback.  ... 
doi:10.1145/2858036.2858237 dblp:conf/chi/CaiIT16 fatcat:2lzawuejhncybl2ucg65xv7zoa

The Silk Road: A Review Essay on Empires of the Silk Road by Christopher I. Beckwith

Thomas D Hall
2010 Cliodynamics  
of empirical evidence that might be used to evaluate those propositions.  ...  Most useful perhaps is the use of thick descriptions to help in theory-building by expanding the topics explained and/or exploring the limits of generality.  ... 
doi:10.21237/c7clio11198 fatcat:qcbb5nzihvcr3bymthmkcllvda

Representation of the East in Western Literature (A Critical Discourse Analysis of the Travelogue Eothen)

Neda Salahshour, Farzad Salahshour
2012 Mediterranean Journal of Social Sciences  
The method applied was van Dijk's ideological square which is used to reveal forms of positive self and negative other.  ...  He mentions a number of writers who he believes depict distorted images of the East in order to satiate their colonizing ends among whom is Kingslake and his travel narrative Eothen (1844).  ...  (Kingslake, 2011, p.5) The above excerpt immediately ends with an evaluative line which tries to highlight the different degrees of activity of the East and West.  ... 
doaj:fe298c785ec5446fad3a827921507ce8 fatcat:nz3lfx54dzd3dnzqjm5c3mteha


Can Eyüp ÇEKİÇ
2020 Türkiye Ortadoğu Çalışmaları Dergisi  
for Turks and Arabs within the Ottoman Empire.  ...  This study aims to expose the ways in which leading officials of the Committee of Union and Progress (the CUP) interpreted, internalized, and questioned the conditions of their mission in Arab lands during  ...  He frequently praises the Arab nation, and clearly notices their loyalty to the Ottoman regime until the end and their usefulness to his service.  ... 
doi:10.26513/tocd.631673 fatcat:m547uvwwljaqfdqkotknshl26u

Analyzing Dynamic Adversarial Training Data in the Limit [article]

Eric Wallace, Adina Williams, Robin Jia, Douwe Kiela
2021 arXiv   pre-print
We argue that running DADC over many rounds maximizes its training-time benefits, as the different rounds can together cover many of the task-relevant phenomena.  ...  We present the first study of longer-term DADC, where we collect 20 rounds of NLI examples for a small set of premise paragraphs, with both adversarial and non-adversarial approaches.  ...  Acknowledgments We thank Max Bartolo, Yixin Nie, Tristan Thrush, Pedro Rodriguez, and the other members of the Dynabench team for their valuable feedback on our crowdsourcing platform and paper.  ... 
arXiv:2110.08514v1 fatcat:rclybepweneyjdbem2cvpmkvxi

Nicole Hahn Rafter — Creating Born Criminals

David Hoogland Noon
2000 Left history  
reproductive peril they appeared to pose.  ...  abandon of laissez-faire economics and the roaming perils of mass democracy.  ... 
doi:10.25071/1913-9632.5466 fatcat:3bnjqitzafd3nhowftwg3dnncy

Automatically Generating Documentation for Lambda Expressions in Java [article]

Anwar Alqaimi, Patanamon Thongtanunam, Christoph Treude
2019 arXiv   pre-print
Our evaluation of LambdaDoc with 23 professional developers shows that they perceive the generated documentation to be complete, concise, and expressive, while the majority of the documentation produced  ...  In this paper, we first present the results of an empirical study to determine how frequently developers of GitHub repositories make use of lambda expressions and how they are documented.  ...  We thank the Government of Saudi Arabia for supporting the first author's studies.  ... 
arXiv:1903.06348v1 fatcat:jxxedqlverftvefjjab6srbyky

Replicating and Scaling up Qualitative Analysis using Crowdsourcing: A Github-based Case Study [article]

Di Chen, Kathryn T. Stolee, Tim Menzies
2017 arXiv   pre-print
That said, they can be used to test the stability and external validity, of the insights gained from a qualitative analysis.  ...  That said, they can guide and define the goals of scalable secondary studies that use (e.g.) crowdsourcing+data mining.  ...  Credit: [45] .employees and outsourcing it to an unde ned (and generally large) network of people in the form of an open call. " Figure 3 : 3 Methodology of Scalable Secondary Studies (MOSSS).  ... 
arXiv:1702.08571v2 fatcat:2uz4ww3vwbfmflmhpjztnfaa6e

The Role of Empowerment in Crowdsourced Customer Service

Stephen Ichatha, Pamela Ellen
2013 Social Science Research Network  
Below are some success stories as told by MTurk's customers: "Acxiom Corporation was able to reduce transcription and outsourcing costs by 50% using Amazon Mechanical Turk" "AOL uses Mechanical Turk to  ...  Procedure In the process flow described in Figure 9 , workers who accepted the HIT were told that the purpose of the study was to evaluate how people in a workforce like Amazon Mechanical Turk (MTurk)  ...  Please answer them to the best of your ability. There are no right or wrong answers. Q4 In general, I see myself as someone who... Strongly  ... 
doi:10.2139/ssrn.2327666 fatcat:2schaqywwrdhfhnk7e53qncidy

Replication Can Improve Prior Results: A GitHub Study of Pull Request Acceptance [article]

Di Chen, Kathyrn Stolee, Tim Menzies
2019 arXiv   pre-print
To test the generality of this approach, the next step in future work is to conduct other studies that extend qualitative studies with crowdsourcing and data mining.  ...  Crowdsourcing and data mining can be used to effectively reduce the effort associated with the partial replication and enhancement of qualitative studies.  ...  ACKNOWLEDGEMENTS The work is partially funded by NSF awards #1506586, #1302169, and #1645136.  ... 
arXiv:1902.04060v1 fatcat:dsg2yqruljbbnlqhipmxha5rgu

From User-Centered to Adoption-Centered Design

Parmit K. Chilana, Andrew J. Ko, Jacob Wobbrock
2015 Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems - CHI '15  
The innovation is a novel contextual help system for the Web, and we reflect on the different methods used to evaluate it and how research insights endure attempted dissemination as a commercial product  ...  We present an in-depth case study of how an HCI research innovation goes through the process of transitioning from a university project to a revenue-generating startup financed by venture capital.  ...  To address this question, we used the Mechanical Turk (mTurk) platform to get access to a large number of users and their help selections.  ... 
doi:10.1145/2702123.2702412 dblp:conf/chi/ChilanaKW15 fatcat:lch7nfgpgjbapcekzit4akczx4

Ousiometrics and Telegnomics: The essence of meaning conforms to a two-dimensional powerful-weak and dangerous-safe framework with diverse corpora presenting a safety bias [article]

P. S. Dodds, T. Alshaabi, M. I. Fudolig, J. W. Zimmerman, J. Lovato, S. Beaulieu, J. R. Minot, M. V. Arnold, A. J. Reagan, C. M. Danforth
2021 arXiv   pre-print
From work emerging through the middle of the 20th century, the essence of meaning has become generally accepted as being well captured by the three orthogonal dimensions of evaluation, potency, and activation  ...  We further show that the PD framework revises the circumplex model of affect as a more general model of state of mind.  ...  OAC-1827314 and NSF award No. 2117345; and from financial support from the Massachusetts Mutual Life Insurance Company and Google Open Source under the Open-Source Complex Ecosystems And Networks (OCEAN  ... 
arXiv:2110.06847v1 fatcat:sexyqz27gnhqbktbvxyaiclc5u

The Contemporary PresidencyObama's Authorization Paradox: Syria and Congress's Continued Relevance in Military Affairs

Douglas L. Kriner
2014 Presidential Studies Quarterly  
A wealth of data from previous interventions in Iraq, Kosovo, Bosnia, and Lebanon suggests that members of both parties who voted to authorize the use of force are much less willing in the future to vote  ...  However, I argue that the decision was more a gambit for political gain than a sincere reevaluation of the scope of presidential war powers.  ...  use the power of the purse to end military actions of which legislators disapproved.  ... 
doi:10.1111/psq.12115 fatcat:ptdxcslhsveflgzxvwb7szf4vm
« Previous Showing results 1 — 15 out of 1,056 results