A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Filters
Evaluating the performance of Automated Text Scoring systems
2015
Proceedings of the Tenth Workshop on Innovative Use of NLP for Building Educational Applications
Preserved Fulltext
Various measures have been used to evaluate the effectiveness of automated text scoring (ATS) systems with respect to a human gold standard. ...
Finally, we identify the most appropriate measures of agreement and present general recommendations for best evaluation practices. ...
Acknowledgments We would like to thank Ted Briscoe for his valuable comments and suggestions, Cambridge English Language Assessment for supporting this research, and the anonymous reviewers for their useful ...
doi:10.3115/v1/w15-0625
dblp:conf/bea/YannakoudakisC15
fatcat:ktj7nwcnkjfvtogm3clgntalhm
Automated MT evaluation metrics and their limitations
2014
Tradumàtica tecnologies de la traducció
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2015; you can also visit the original URL.
The file type is application/pdf.
Automated evaluation of MT characterizes performance of MT systems on specific text or a corpus. ...
Automated scores are expected to correlate with certain parameters of MT quality scored by human evaluators, such as adequacy of fluency of translation. ...
The main objective of methods and tools for automated evaluation of MT is to compute numerical scores, which characterize the 'quality', or the level of performance of specific Machine Translation systems ...
doi:10.5565/rev/tradumatica.70
fatcat:lkr27o633vc3vla76gk7mwtpy4
Comparative Study on Feature-Based Scoring Using Vector Space Modelling System
2021
Mathematical Problems in Engineering
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Considering the potential of the automated scoring system, there is further a need to refine and develop the existing system. The paper goes through the state of the art. ...
This paper shows the importance of automated scoring (AS) and that it is better than human graders in terms of degree of reproducibility. ...
Omar Cheikhrouhou thanks Taif University for its support under the project Taif University Researchers Supporting Project (no. TURSP-2020/55), Taif University, Taif, Saudi Arabia. ...
doi:10.1155/2021/9946573
fatcat:g634sxeylfa35g57c5w6nk5p7y
CaseSummarizer: A System for Automated Summarization of Legal Texts
2016
International Conference on Computational Linguistics
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
It is evaluated using ROUGE and human scoring against several other summarization systems, including summary text and feedback provided by domain experts. ...
Attorneys, judges, and others in the justice system are constantly surrounded by large amounts of legal text, which can be difficult to manage across many cases. ...
Acknowledgments The authors would like to thank each of the domain experts who provided high-quality summaries and ratings on the systems, as well as all those who gave feedback on CaseSummarizer's various ...
dblp:conf/coling/PolsleyJH16
fatcat:y3dcjxu5dvg5djhph5lndbd76e
Open Problems in Indonesian Automatic Essay Scoring System
2018
International Journal of Engineering & Technology
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
This paper presents open problems in Indonesian Scoring System. The previous study exposes the comparison of several similarity metrics on automated essay scoring in Indonesian. ...
The study also evaluates the stemming approach for the system performance. The difference between all methods between using stemming or not is around 4-9%. ...
The authors would like to thanks to the institution, especially UPT P2M for the full support and funding. ...
doi:10.14419/ijet.v7i4.44.26974
fatcat:k43k464lgncfnn2okuz36fr474
Automated Evaluation ofTelugu Text Essays Using Latent Semantic Analysis
2021
Turkish Journal of Computer and Mathematics Education
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Automated systems are increasingly required to help students write essays. In the field of educational assessment technology, automated test evaluation is becoming more and more common. ...
Language skills; the structure of the essay and the contents that fit the subject are the principal requirements for evaluating the essays. ...
We can only describe AEE as a means of automated computer evaluation of the written prose [10] . Evaluation means that the computer system will do the job of scored or assigned a number to an essay. ...
doi:10.17762/turcomat.v12i5.2267
fatcat:h3dly3tmbrgv5i4e7llqdbruee
SURVEY PAPER ON SMART ESSAY GRADER
2021
International Journal of Engineering Applied Sciences and Technology
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
In order to optimize Human-Machine agreement for automatic evaluation of textual summaries or essays, automated essay grading has been a research field. ...
, in order to rate the essay with the highest level of accuracy. ...
The results show that the system outperforms a strong baseline in automated essay scoring and achieves state-of-the-art success. ...
doi:10.33564/ijeast.2021.v05i11.017
fatcat:g4hyhad7izerdfgldj4o4fveyy
Learning to Compare for Better Training and Evaluation of Open Domain Natural Language Generation Models
2020
PROCEEDINGS OF THE THIRTIETH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE AND THE TWENTY-EIGHTH INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Training with the proposed metric yields better performance in human evaluation, which further demonstrates the effectiveness of the proposed model. ...
We also propose to evaluate the model-level quality of NLG models with sample-level comparison results with skill rating system. ...
evaluator with n human-annotated scores, 3) score-based human evaluation facilitates the evaluation of correlation scores, and 4) as all other metrics do not perform pairwise comparison, using pairwise ...
doi:10.1609/aaai.v34i05.6521
fatcat:xbkgmohbsvayzoiecslendjley
Learning to Compare for Better Training and Evaluation of Open Domain Natural Language Generation Models
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Training with the proposed metric yields better performance in human evaluation, which further demonstrates the effectiveness of the proposed model. ...
We also propose to evaluate the model-level quality of NLG models with sample-level comparison results with skill rating system. ...
evaluator with n human-annotated scores, 3) score-based human evaluation facilitates the evaluation of correlation scores, and 4) as all other metrics do not perform pairwise comparison, using pairwise ...
arXiv:2002.05058v1
fatcat:s5ykq6n6vncxdgsshunnpfefza
DOMAIN SPECIFIC AUTOMATED ESSAY SCORING USING CLOUD BASED NLP API
2021
International journal of computer science and mobile computing
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
These topic graphs are either to be created by the teacher or automatically generated by scanning a certain amount of sample texts in the subject area. ...
The paper presents a methodology and an application framework (PUAnalyzeThis) that makes use of MeaningCloud API to automatically extract entities, concepts, relations, etc. and calculate scores and grades ...
The performance of the system-generated scores is compared with the human scores using Pearson correlation. The results show that system and human scores correlatewith each other. Dong et al. ...
doi:10.47760/ijcsmc.2021.v10i10.006
fatcat:czlijofpcnhtjj67bqxj3efidu
Coherence Modeling for the Automated Assessment of Spontaneous Spoken Responses
2013
North American Chapter of the Association for Computational Linguistics
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
An analysis on the annotated corpus shows that the prediction accuracy for human holistic scores of an automated speech scoring system can be improved by around 10% relative after the addition of the coherence ...
Further experiments indicate that a weighted F-Measure of 73% can be achieved for the automated prediction of the coherence scores. ...
Acknowledgments The authors wish to express our thanks to the discourse annotators Melissa Lopez and Matt Mulholland for their dedicated work and our colleagues Jill Burstein and Slava Andreyev for their ...
dblp:conf/naacl/WangEZ13
fatcat:yj66iwkjobexratb4jzp2o3eha
Speech- and Text-driven Features for Automated Scoring of English Speaking Tasks
2017
Proceedings of the Workshop on Speech-Centric Natural Language Processing
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Although adding any single category of acoustic features to the text-only system on its own does not significantly improve performance, adding all acoustic features together does yield a small but significant ...
We combine features from a text-only content scoring system originally designed for written responses with several categories of acoustic features. ...
We also thank Matt Mulholland for help with processing the data. ...
doi:10.18653/v1/w17-4609
dblp:conf/emnlp/LoukinaMC17
fatcat:nzdlk3bji5hevppwa7lsi36tha
Bring It on! Challenges Encountered While Building a Comprehensive Tutoring System Using ReaderBench
[chapter]
2018
Lecture Notes in Computer Science
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
This paper describes the challenges encountered while implementing an automated evaluation workflow and adopting solutions for increasing performance of the tutoring system. ...
Intelligent Tutoring Systems (ITSs) are aimed at promoting acquisition of knowledge and skills by providing relevant and appropriate feedback during students' practice activities. ...
This research was partially supported by the 644187 EC H2020 Realising an Applied Gaming Eco-system (RAGE) project, the FP7 2008-212578 LTfLL project, the Department of Education, Institute of Education ...
doi:10.1007/978-3-319-93843-1_30
fatcat:uxsua3sscnb53ererrgahmseue
Contactless Academia – The Case for Automated Essay Scoring (AES) System in COVID 19 Pandemic
2021
Current Journal of Applied Science and Technology
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
An invaluable measure worth considering is the inclusion of 'Automated Essay Scoring' (AES) system in the mitigation toolkits for higher institutions of learning. ...
The outcome of reviewed literatures varied on suitability of AES in scoring essay task in Higher Institution of Learning. ...
Giles investigated the impact of an automated grading system (SAGrader) on student's performance. ...
doi:10.9734/cjast/2021/v40i431292
fatcat:hu2tfmidrzczrodg5bta7jsaiu
Effectiveness of an Online Automated Evaluation and Feedback System in an Introductory Computer Literacy Course
2014
Eurasia Journal of Mathematics, Science and Technology Education
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
The purpose of this study was to investigate the effectiveness of an online automated evaluation and feedback system that assessed students' word processing assignments prepared with Microsoft Office Word ...
in which the experimental group students used the online automated evaluation and feedback system to get feedback on their assignments, and the control group students didn't receive any feedback. ...
Majority of the students found the automated evaluation system excellent and they believed it perfectly evaluated their codes and corrected the errors. ...
doi:10.12973/eurasia.2014.1062a
fatcat:h4gwagjjqnfvrm4kms6cid4qw4
« Previous
Showing results 1 — 15 out of 112,759 results