31,070 Hits in 4.2 sec

Sensory Panel Performance Evaluation—Comprehensive Review of Practical Approaches

László Sipos, Ákos Nyitrai, Géza Hitka, László Ferenc Friedrich, Zoltán Kókai
2021 Applied Sciences  
A practical overview is given on the available solutions, methods, protocols and software relating to all major panelist and panel measure indices (agreement, discrimination, repeatability, reproducibility  ...  Sensory assessors determine the result of sensory analysis; therefore, investigation of panel performance is inevitable to obtain well-established results.  ...  The selection and training methods to be employed depend on the tasks to intend for the selected assessors and expert sensory assessors [3] . In the present paper we focus on these assessor types.  ... 
doi:10.3390/app112411977 fatcat:tezhotpahndclpqvczpny6krbu

Aggregation of Multiple Judgments for Evaluating Ordered Lists [chapter]

Hyun Duk Kim, ChengXiang Zhai, Jiawei Han
2010 Lecture Notes in Computer Science  
Experiment results on ordering sentences for text summarization show that all the three new methods outperform the state of the art average correlation methods in terms of discriminativeness and robustness  ...  Among the three proposed methods, the frequent sequential pattern-based method performs the best due to the flexible modeling of agreements and disagreements among human experts at various levels of granularity  ...  In WCA, the weight on each assessor is based on the overall agreement between the assessor and other assessors.  ... 
doi:10.1007/978-3-642-12275-0_17 fatcat:tkb677v645c7pbjz662ag6sb5q

The effect of threshold priming and need for cognition on relevance calibration and assessment

Falk Scholer, Diane Kelly, Wan-Ching Wu, Hanseul S. Lee, William Webber
2013 Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval - SIGIR '13  
We examine the effect that threshold priming, seeing varying degrees of relevant documents, has on people's calibration of relevance.  ...  Showing documents to assessors in different orderings, however, may lead to different assessment outcomes.  ...  (Topic was included in the analysis due to the strong topic effect on agreement between assessors noted in Section 4.2.)  ... 
doi:10.1145/2484028.2484090 dblp:conf/sigir/ScholerKWLW13 fatcat:klp6et5sefachjfirzlyagjdqu

Assessing domain based teaching skills of postgraduates after a workshop on effective lesson plan

Sujoy Ray, Amita Ray, Arun Gopi, Dhayakani Selvakumar
2018 Journal of Education Technology in Health Sciences  
Kappa statistics was used to determine the agreement between the three assessors.  ...  The assessors found 70-100% use of the domains of a lesson plan which resulted in effective teaching.  ...  One explained the domains or components of an effective lesson plan and the second gave instructions on creating an effective power point presentation.  ... 
doi:10.18231/2393-8005.2018.0017 fatcat:daze3ghwjjgudny4wfbr4w4riq

On the Relation Between Assessor's Agreement and Accuracy in Gamified Relevance Assessment

Olga Megorskaya, Vladimir Kukushkin, Pavel Serdyukov
2015 Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval - SIGIR '15  
One of the possible ways of controlling the quality is monitoring inter-assessor agreement level. But does the agreement level really reflect the quality of assessor's judgments?  ...  Namely, we estimate the correlation between assessors' accuracy and agreement in the scope of several workflow designs and investigate which specific workflow features influence the accuracy of judgments  ...  Second, we focus on the impact of discussions on assessors' accuracy, agreement, experience and the overall effectiveness of a workflow.  ... 
doi:10.1145/2766462.2767727 dblp:conf/sigir/MegorskayaKS15 fatcat:vfl33ksuajb7td2xfafjxrnx5i

Clustering around latent variables approach to detect panel disagreement in three–way tea sensory evaluation

DR Fernando, S Samita, TUS Peiris
2020 International Journal of Scientific and Research Publications (IJSRP)  
However, by examining assessor weights it was possible to identify assessors those who were in agreement and those who were in disagreement for each component.  ...  Therefore, CLV3W method is a useful method to detect disagreement between assessors, especially in 3-way data, and it can effectively be used when selecting assessors for sensory evaluation.  ...  panel on one latent weighting strategy effectively to identify assessors who need more training on which subset of attributes.  ... 
doi:10.29322/ijsrp.10.06.2020.p10291 fatcat:fjvfga564zbklgcovh6dhnwmta

A Methodology for Evaluating Aggregated Search Results [chapter]

Jaime Arguello, Fernando Diaz, Jamie Callan, Ben Carterette
2011 Lecture Notes in Computer Science  
An extensive user study with 13 verticals confirms that, when users prefer one presentation of results over another, our metric agrees with the stated preference.  ...  While most prior work focuses on deciding which verticals to present, the task of deciding where in the Web results to embed the vertical results has received less attention.  ...  Assessors may require more instruction in order to improve agreement on presentation-pairs. Alternatively, more than 4 assessors may be required to see greater convergence.  ... 
doi:10.1007/978-3-642-20161-5_15 fatcat:3qeydavmvbh6zd47wrxdtrptv4

Relevance judgments between TREC and Non-TREC assessors

Azzah Al-Maskari, Mark Sanderson, Paul Clough
2008 Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval - SIGIR '08  
Several factors contributed to differences in the agreements: the number of retrieved relevant documents; the number of relevant documents judged; system effectiveness per topic and the ranking of relevant  ...  This paper investigates the agreement of relevance assessments between official TREC judgments and those generated from an interactive IR experiment.  ...  Users were given 7 minutes for each topic, and a Latin-Square arrangement utilised to distribute the order of the topics amongst users (to reduce the effects of topic order on results).  ... 
doi:10.1145/1390334.1390450 dblp:conf/sigir/Al-MaskariSC08 fatcat:fz26ofjcvbfnfp3fnfs6j5i5py

Cross-cultural perception of female facial appearance: A multi-ethnic and multi-centre study

Rainer Voegeli, Rotraut Schoop, Elodie Prestat-Marquis, Anthony V. Rawlings, Todd K. Shackelford, Bernhard Fink, Piotr Sorokowski
2021 PLoS ONE  
Linear mixed model analysis revealed main and interaction effects of assessor ethnicity, assessor gender, and photographed participant ("face") ethnicity on age, attractiveness, and health assessments.  ...  Thus, differences in judgments of female facial appearance depend on the ethnicity of the photographed person, the ethnicity of the assessor, and whether the assessor is female or male.  ...  Table 3 reports the main effects and interactions of assessor ethnicity, assessor gender and face ethnicity on perception of facial age and Tables 4-6 present descriptive statistics for age, attractiveness  ... 
doi:10.1371/journal.pone.0245998 pmid:33481957 fatcat:nzmiwqt3i5aydangkr4jkst42i

Reliability of the cervical vertebrae maturation (CVM) method

A. Predko-Engel, M. Kaminek, K. Langova, P. Kowalski, P. S. Fudalej
2015 Bratislava Medical Journal  
Kappa statistics (with lower limits of 95% confi dence intervals (CI)) and proportion of complete agreement on staging was used to evaluate intra-and inter-assessor agreement.  ...  Results: The mean weighted kappa for intra-assessor agreement was 0.44 (range: 0.30-0.64; range of lower limits of 95% CI: 0.12-0.48) and for inter-assessor agreement was 0.28 (range: -0.01-0.58; range  ...  Experienced assessors demonstrated similar intra-assessor agreement as Inexperienced ones: 0.45 versus 0.44, respectively.  ... 
doi:10.4149/bll_2015_043 fatcat:yzwqftwl4fhkdebd47msekdno4

Comparison of self-administered University of California, Los Angeles, shoulder score with traditional University of California, Los Angeles, shoulder score completed by clinicians in assessing the outcome of rotator cuff surgery

Jennifer A. Coghlan, Simon N. Bell, Andrew Forbes, Rachelle Buchbinder
2008 Journal of shoulder and elbow surgery  
The patients were then evaluated by assessor | (the surgeon who performed the operations} and assessor 2 (an experienced research assistant) in ran- dom order based on computer-generated random numbers  ...  Present all the time and unbearable; needing strong medication frequently one number in each section which best describes your b.  ... 
doi:10.1016/j.jse.2007.11.020 pmid:18372196 fatcat:vl76l73dlnexxlq645vg72bk6i

Noise and accustomation: A pilot study of trained assessors' olfactory performance

Johanna Trautmann, Lisa Meier-Dinkel, Jan Gertheiss, Daniel Mörlein, Manabu Sakakibara
2017 PLoS ONE  
The results indicate that sensory quality control can be conducted in a manufacturing environment with constant noise without diminishing the assessors' performance.  ...  Two groups of panelists were selected: a University panel usually working under silent conditions and an abattoir panel usually working on the slaughter line with abattoir noise.  ...  Samples were presented in a randomized order for each assessor/each condition and identified with a three digit code.  ... 
doi:10.1371/journal.pone.0174697 pmid:28380041 pmcid:PMC5381871 fatcat:dczsmmsv55h2nipsrs3kyc5mgi

Reliable Assessors of Infant Cranial Asymmetry in Child Health Care

Freda Lennartsson, Göran Wennergren, Per Nordin
2015 Open Nursing Journal  
Agreement matrices were devised to illustrate assessor agreement based on both type and degree of cranial asymmetry.  ...  Agreement based on degree of asymmetry was analyzed by calculating AC2 using quadratic weights.  ...  When asymmetry was present according to the reference rater, the assessors detected it, but there was no complete agreement among assessors for any of the five infants with asymmetry.  ... 
doi:10.2174/1874434601509010033 pmid:26401172 pmcid:PMC4578139 fatcat:radgisiwejdblcqv32gwtjasni

Does Thirty-Minute Standardised Training Improve the Inter-Observer Reliability of the Horse Grimace Scale (HGS)? A Case Study

Francesca Dai, Mattew Leach, Amelia Mari MacRae, Michela Minero, Emanuela Dalla Costa
2020 Animals  
The aim of the study was to evaluate whether a 30-minute standardised training program on HGS is effective at improving the agreement between observers with no horse experience and when compared to an  ...  Using valid assessment methods to identify pain in horses is of a clear importance; however, the reliability of the assessment is highly dependent on the assessors' ability to use it.  ...  A training program aiming to improve the accuracy of pain evaluation by new assessors should be developed in order to improve their inter-observer reliability [6, 7] .  ... 
doi:10.3390/ani10050781 pmid:32365927 pmcid:PMC7277819 fatcat:wo6tgajilbahph4dfhrpmx5qoi

Determining priority for joint replacement: comparing the views of orthopaedic surgeons and other professionals

Andrea J Curtis, Rory Wolfe, Colin O H Russell, Barry G Elliott, John A L Hart, John J McNeil
2011 Medical Journal of Australia  
Acknowledgements: We thank the patients, research nurses, recruiting orthopaedic surgeons, and members of the assessor panel. The study was funded by the Victorian Department of Human Services.  ...  Patient interviews were summarised and presented to assessors who ranked groups of eight patients in order of their perceived urgency for surgery.  ...  For each group of eight patients, there was moderate agreement on overall urgency rankings between the two groups of assessors.  ... 
doi:10.5694/mja10.11052 fatcat:eeu4ef6nlzer7dbjo323dgfkfi
« Previous Showing results 1 — 15 out of 31,070 results