34,356 Hits in 4.2 sec

Crowdsourcing Quality of Experience Experiments [chapter]

Sebastian Egger-Lampl, Judith Redi, Tobias Hoßfeld, Matthias Hirth, Sebastian Möller, Babak Naderi, Christian Keimel, Dietmar Saupe
2017 Lecture Notes in Computer Science  
The authors want to thank Schloss Dagstuhl Leibniz-Zentrum für Informatik, the participants of Dagstuhl Seminar 15481 Evaluation in the Crowd: Crowdsourcing and Human-Centred Experiments as well as the  ...  Qualinet members that participated in the creation of Best Practices and Recommendations for Crowdsourced QoE -Lessons learned from the Qualinet Task Force [38] .  ...  Nevertheless, crowdsourcing of QoE experiments also faces certain challenges.  ... 
doi:10.1007/978-3-319-66435-4_7 fatcat:mkhhxbmtpzhqzcawfep7m7v5oi

Robust evaluation for quality of experience in crowdsourcing

Qianqian Xu, Jiechao Xiong, Qingming Huang, Yuan Yao
2013 Proceedings of the 21st ACM international conference on Multimedia - MM '13  
Strategies exploiting crowdsourcing are increasingly being applied in the area of Quality of Experience (QoE) for multimedia.  ...  The scheme shows that the removal of outliers in crowdsourcing experiments would be helpful for purifying data and could provide us with more reliable results.  ...  A novel method for robust evaluation of QoE is proposed to deal with incomplete and imbalanced data in crowdsourcing experiments.  ... 
doi:10.1145/2502081.2502083 dblp:conf/mm/XuXHY13 fatcat:fxdnoh2jknb7dkx4egcc5k6hpe

Crowdsourcing based subjective quality assessment of adaptive video streaming

M. Shahid, J. Sogaard, J. Pokhrel, K. Brunnstrom, K. Wang, S. Tavakoli, N. Gracia
2014 2014 Sixth International Workshop on Quality of Multimedia Experience (QoMEX)  
Besides lab-based subjective experiments, crowdsourcing based subjective assessment of video quality is gaining popularity as an alternative method.  ...  Perceptual quality experiments that involve humans are considered to be the most valid method of the assessment of QoE.  ...  INTRODUCTION Interest in quality of experience (QoE) of video services is growing due to increasing usage of videos over networks, such as the portion of video data in mobile networks is expected to exceed  ... 
doi:10.1109/qomex.2014.6982289 dblp:conf/qomex/ShahidSPBWTG14 fatcat:bjzwewu76rbwlk2xrr5lc3trrq

Quality of Crowdsourced Data on Urban Morphology—The Human Influence Experiment (HUMINEX)

Benjamin Bechtel, Matthias Demuzere, Panagiotis Sismanidis, Daniel Fenner, Oscar Brousse, Christoph Beck, Frieke Van Coillie, Olaf Conrad, Iphigenia Keramitsoglou, Ariane Middel, Gerald Mills, Dev Niyogi (+3 others)
2017 Urban Science  
In this study, we investigated the quality of 94 crowdsourced training datasets for ten cities, generated by 119 students from six universities.  ...  This is achieved through crowdsourcing, which we define here as the collection of data by a bounded crowd, composed of students.  ...  Acknowledgments: Most of all we thank the participating students, namely: Alex Apostolakis, Lien Arnalsteen,  ... 
doi:10.3390/urbansci1020015 fatcat:677no5ighvamzcar6hl5b3s6ka

Journey through the crowd: Best practices and recommendations for crowdsourced QoE

Tobias Hosfeld, Judith Redi
2015 2015 Seventh International Workshop on Quality of Multimedia Experience (QoMEX)  
However, there are common mistakes and pitfalls which are to be kept in mind for ensuring the reliability of such data.  ...  Crowdsourcing-based QoE testing is easy to implement and gives quick results at low costs; as a result crowdsourced QoE data is nowadays popular.  ...  For quality assessment of multimedia services and applications, crowdsourcing opens new perspectives, by moving subjective testing into the crowd, resulting in larger diversity of the test subjects, reduced  ... 
doi:10.1109/qomex.2015.7148150 dblp:conf/qomex/HosfeldR15 fatcat:fimhskoaxbaarjeq24v7a5i4me

On subjective quality assessment of adaptive video streaming via crowdsourcing and laboratory based experiments

Jacob Søgaard, Muhammad Shahid, Jeevan Pokhrel, Kjell Brunnström
2016 Multimedia tools and applications  
On subjective quality assessment of adaptive video streaming via crowdsourcing and laboratory based experiments.  ...  Besides laboratory based subjective experiments, crowdsourcing based subjective assessment of video quality is gaining popularity as an alternative  ...  An interesting extension of this work would be to analyze the demographics of the viewers from the experiments, especially the crowdsourcing experiment, and investigate any correlations between the demographics  ... 
doi:10.1007/s11042-016-3948-3 fatcat:nyw5fwzx6zdwzo64ure46vdrky

Quality of Experience of Web-based Adaptive HTTP Streaming Clients in Real-World Environments using Crowdsourcing

Benjamin Rainer, Christian Timmerer
2014 Proceedings of the 2014 Workshop on Design, Quality and Deployment of Adaptive Video Streaming - VideoNext '14  
The aim of this paper is to compare the QoE performance of existing DASH-based Web clients within real-world environments using crowdsourcing.  ...  Based on a predefined content configuration, which is comparable among the clients, we run a crowdsourcing campaign to determine the QoE of each implementation in order to determine the current state-of-the-art  ...  ACKNOWLEDGEMENTS This work was supported in part by the EC in the context of the SocialSensor (FP7-ICT-287975) and QUALINET (COST IC 1003) projects and partly performed in the Lakeside Labs research cluster  ... 
doi:10.1145/2676652.2676656 fatcat:gyqoubwpnnhflp2iwii3zog7eq

Is one second enough? Evaluating QoE for inter-destination multimedia synchronization using human computation and crowdsourcing

Benjamin Rainer, Stefan Petscharnig, Christian Timmerer, Hermann Hellwagner
2015 2015 Seventh International Workshop on Quality of Multimedia Experience (QoMEX)  
Our results show a significant decrease in overall Quality of Experience (QOE) at an asynchronism level of 750ms.  ...  Inter-Destination Multimedia Synchronization (IDMS) is the tool of choice in order to enable users a high-quality multimedia experience.  ...  In order to provide the same Quality of Experience (QoE) as if the users were in the same room in front of the TV, the playback of the multimedia content among the geographically distributed participants  ... 
doi:10.1109/qomex.2015.7148107 dblp:conf/qomex/RainerPTH15 fatcat:wfglqqfap5bp3o7ovy3alrrium


Matteo Varvello, Jeremy Blackburn, David Naylor, Konstantina Papagiannaki
2016 Proceedings of the 12th International on Conference on emerging Networking EXperiments and Technologies - CoNEXT '16  
In this paper we present Eyeorg, a platform for crowdsourcing web quality of experience measurements.  ...  , and 3) assess the impact of online advertisements and ad blockers on user experience.  ...  The European Commission is not responsible for any use that may be made of that information.  ... 
doi:10.1145/2999572.2999590 dblp:conf/conext/VarvelloBNP16 fatcat:gq6l4vbsrzdstfkr7pgc5kbfs4

On the state of reporting in crowdsourcing experiments and a checklist to aid current practices [article]

Jorge Ramírez, Burcu Sayin, Marcos Baez, Fabio Casati, Luca Cernuzzi, Boualem Benatallah, Gianluca Demartini
2021 arXiv   pre-print
In this paper, we examine the current state of reporting of crowdsourcing experiments and offer guidance to address associated reporting issues.  ...  While several guidelines inform researchers in these choices, guidance of how and what to report from crowdsourcing experiments has been largely overlooked.  ...  , the quality control mechanisms used to guard the quality of the results, and the outcome of the experiment.  ... 
arXiv:2107.13519v2 fatcat:wxwom43pv5dnbh6zho2knr7tgm

Information Visualization Evaluation Using Crowdsourcing

R. Borgo, L. Micallef, B. Bach, F. McGee, B. Lee
2018 Computer graphics forum (Print)  
Visualization researchers have been increasingly leveraging crowdsourcing approaches to overcome a number of limitations of controlled laboratory experiments, including small participant sample sizes and  ...  We tagged each experiment along 36 dimensions that we identified for crowdsourcing experiments.  ...  Purchase, as well as all of the participants for the interesting discussions that motivated this work. We also thank the staff at Schloss Dagstuhl for hosting this seminar.  ... 
doi:10.1111/cgf.13444 fatcat:z7w4vfws5zbrnkapfztmap3ipu

Creation of Reliable Relevance Judgments in Information Retrieval Systems Evaluation Experimentation through Crowdsourcing: A Review

Parnia Samimi, Sri Devi Ravana
2014 The Scientific World Journal  
In order to have a successful crowdsourcing experiment, the relevance judgment tasks should be designed precisely to emphasize quality control.  ...  experiment.  ...  Conflict of Interests The authors declare that there is no conflict of interests regarding the publication of this paper. Acknowledgment  ... 
doi:10.1155/2014/135641 pmid:24977172 pmcid:PMC4055211 fatcat:qfbavfc45jfmzp4yisqyedx2o4

Crowdsourcing Versus the Laboratory: Towards Human-Centered Experiments Using the Crowd [chapter]

Ujwal Gadiraju, Sebastian Möller, Martin Nöllenburg, Dietmar Saupe, Sebastian Egger-Lampl, Daniel Archambault, Brian Fisher
2017 Lecture Notes in Computer Science  
We would like to thank Dagstuhl for facilitating the seminar (titled, 'Evaluation in the Crowd: Crowdsourcing and Human-Centred Experiments') that brought about this collaboration.  ...  Part of this work (Sect. 4) was supported by the German Research Foundation (DFG) within project A05 of SFB/Transregio 161.  ...  of crowdsourcing for human-centered experiments is quality control and the reliability of participants.  ... 
doi:10.1007/978-3-319-66435-4_2 fatcat:wqthlc7ohjc6td3wgfdmkyca7q

Challenges and strategies for running controlled crowdsourcing experiments [article]

Jorge Ramírez, Marcos Baez, Fabio Casati, Luca Cernuzzi, Boualem Benatallah
2020 arXiv   pre-print
change the outcome of experiments.  ...  the challenges of running controlled experiments in the "wild".  ...  However, it also offers features to automatically manage pricing, quality control, and scheduling of crowdsourcing tasks.  ... 
arXiv:2011.02804v1 fatcat:o4zj7d2sona63jdgthim53erqi

The Value of Crowdsourcing for Complex Problems: Comparative Evidence from Software Developed By the Crowd And Professionals

Abhishek Tripathi
2016 Journal of Computer Science Applications and Information Technology  
User Experience In this research, we define perceived quality from the user experience perspective.  ...  Perceived quality of the product is measured in terms of pragmatic quality, hedonic quality stimulation, and hedonic quality identification.  ... 
doi:10.15226/2474-9257/1/1/00101 fatcat:fmuqy5qavvc6pmgvjalyjnuewi
« Previous Showing results 1 — 15 out of 34,356 results