21,382 Hits in 9.7 sec

A Comparison of Greedy and Optimal Assessment of Natural Language Student Input Using Word-to-Word Similarity Metrics

Vasile Rus, Mihai C. Lintean
2012 Workshop on Innovative Use of NLP for Building Educational Applications  
We present in this paper a novel, optimal semantic similarity approach based on word-to-word similarity metrics to solve the important task of assessing natural language student input in dialogue-based  ...  The optimal matching is guaranteed using the sailor assignment problem, also known as the job assignment problem, a well-known combinatorial optimization problem.  ...  To the best of our knowledge, nobody proposed an optimal solution based on the principle of compositionality and word-to-word similarity metrics for the student input assessment problem.  ... 
dblp:conf/bea/RusL12 fatcat:r6pwhbtjivgd7exfajzwcjuwve

Performance Evaluation of LSA, NMF and ILSA in Electronic Assessment of Free Text Document

M. M. Rufai, A. O. Afolabi, O. D. Fenwa, F. A. Ajala
2021 Asian Journal of Research in Computer Science  
One hundred students' responses to a test question in an introductory artificial intelligence course were used.  ...  using metrics, Term Similarity, Precision, Recall and F-measure functions, Mean divergence, Assessment Accuracy and Adequacy in Semantic Representation.  ...  Our approach uses Term similarity to confirm two naturally similar terms and two naturally dis-similar terms.  ... 
doi:10.9734/ajrcos/2021/v9i130214 fatcat:dxsxunro5bf53orhgk4ngooc3y

Reference-based Metrics can be Replaced with Reference-less Metrics in Evaluating Grammatical Error Correction Systems

Hiroki Asano, Tomoya Mizumoto, Kentaro Inui
2017 International Joint Conference on Natural Language Processing  
Further, we empirically show that a reference-less metric that combines fluency and meaning preservation with grammaticality provides a better estimate of manual scores than that of commonly used reference-based  ...  To address this problem, a referenceless approach has recently emerged; however, previous reference-less metrics that only consider the criterion of grammaticality, have not worked as well as reference-based  ...  To assess how much of the meaning of an original sentence is preserved in a revision, one can consider the use of an evaluation metric devised in the MT field.  ... 
dblp:conf/ijcnlp/AsanoMI17 fatcat:mkfhamzxmjcfpmh7jtxdm723fa

Automatic Short Answer Grading with SemSpace Sense Vectors and MaLSTM

Cagatay N. Tulu, Ozge Ozkaya, Umut Orhan
2021 IEEE Access  
Also, the proposed approach has been tested as a case study using a specific dataset (CU-NLP) created from the exam of the "Natural Language Processing" course in the Computer Engineering Department of  ...  Automatic assessment of exams is widely preferred by educators than multiple-choice exams because of its efficiency in measuring student performance, lack of subjectivity when evaluating student response  ...  In the training of the model, the mean square error was preferred as the loss function, Adam optimizer for optimization, and mean absolute error (MAE) as the success metric.  ... 
doi:10.1109/access.2021.3054346 fatcat:w2clofjemffqncournevrizjtq

BERT: A Review of Applications in Natural Language Processing and Understanding [article]

M. V. Koroteev
2021 arXiv   pre-print
This survey will be useful to all students and researchers who want to get acquainted with the latest advances in the field of natural language text analysis.  ...  The paper describes the mechanism of operation of this model, the main areas of its application to the tasks of text analytics, comparisons with similar models in each task, as well as a description of  ...  However, the field of natural language word processing remains under-explored due to the specific nature of the input data.  ... 
arXiv:2103.11943v1 fatcat:e3ojyslcine6tmhenayiglxywa

UPB at GermEval-2020 Task 3: Assessing Summaries for German Texts using BERTScore and Sentence-BERT

Andrei Paraschiv, Dumitru-Clementin Cercel
2020 Swiss Text Analytics Conference  
We compare two BERT-based metrics, Sentence-BERT and BERTScore, to automatically evaluate the quality of summaries in the German language.  ...  Our lowest error rate achieved was 31.9925, ranking us in 4th place out of 6 participating teams. For almost twenty years, BLEU (Papineni et al., 2002) , ROUGE (Lin, 2004), and METEOR  ...  (Banerjee and Lavie, 2005) are the most used metrics to assess summaries.  ... 
dblp:conf/swisstext/ParaschivC20 fatcat:tzuqdmsjnnhxzinlpd4oj5n6z4

Improving Assessment of Students through Semantic Space Construction

Roberto Pirrone, Giuseppe Russo, Vincenzo Cannella
2009 2009 International Conference on Complex, Intelligent and Software Intensive Systems  
In this respect, the assessment problem is strictly connected to natural language understanding. The preliminary step is indeed to understand questions and replies of the student.  ...  Assessment is one of the hardest tasks an Intelligent Tutoring System has to perform.  ...  In our vision, an enhanced natural language understanding is a powerful tool to perform assessment of students' skills.  ... 
doi:10.1109/cisis.2009.137 dblp:conf/cisis/PirroneRC09 fatcat:v6d3tyomejebxkgn4ocgve5ht4

MTLHealth: A Deep Learning System for Detecting Disturbing Content in Student Essays [article]

Joseph Valencia, Erin Yao
2021 arXiv   pre-print
Graders must take great care to identify cases like these and decide whether to alert authorities on behalf of students who may be in danger.  ...  Essay submissions to standardized tests like the ACT occasionally include references to bullying, self-harm, violence, and other forms of disturbing content.  ...  Acknowledgments The authors would like to acknowledge John Whitmer and Brian LaMure of ACT for providing access to essential computing resources.  ... 
arXiv:2103.04290v2 fatcat:6bq6gir4qfhspg53oa4p4nugnm

Assessing the Quality of Unstructured Data: An Initial Overview

Cornelia Kiefer
2016 Lernen, Wissen, Daten, Analysen  
We define data quality of unstructured data via (1) the similarity of the input data to the data expected by these consumers of unstructured data and via (2) the similarity of the input data to the data  ...  Finally, we propose automatically measurable indicators for assessing the quality of unstructured text data and give hints towards an implementation.  ...  The authors would like to thank the German Research Foundation (DFG) for financial support of this project as part of the Graduate School of Excellence advanced Manufacturing Engineering (GSaME) at the  ... 
dblp:conf/lwa/Kiefer16 fatcat:xcjr27bimfcxxcmved65fwvmn4

Image to Language Understanding: Captioning approach [article]

Madhavan Seshadri, Malavika Srikanth, Mikhail Belov
2020 arXiv   pre-print
Such an approach is a combination of Computer Vision and Natural Language techniques which is a hard problem to solve.  ...  Representation of such a format in Natural Language has a huge variety of applications such as helping the visually impaired etc.  ...  LSTMs are widely used in language modelling tasks. V. METRICS In order to quantitatively assess our models' output image captions, we used 4 benchmark NLP metrics: BLEU, GLEU, METEOR, and ROUGE.  ... 
arXiv:2002.09536v1 fatcat:ac72qj5wzzb67hbgziy5tpz25u

Identifying Similar Test Cases That Are Specified in Natural Language [article]

Markos Viggiato, Dale Paas, Chris Buzon, Cor-Paul Bezemer
2021 arXiv   pre-print
Our approach uses a combination of text embedding, text similarity and clustering techniques to identify similar test cases.  ...  Through an evaluation in an industrial setting, we showed that our approach achieves a high performance to cluster test steps (an F-score of 87.39%) and identify similar test cases (an F-score of 83.47%  ...  ACKNOWLEDGMENTS The research reported in this article has been supported by Prodigy Education and the Natural Sciences and Engineering Research Council of Canada under the Alliance Grant project ALLRP  ... 
arXiv:2110.07733v1 fatcat:lak6mxytendk5fllcpgvyh3ane

Speech-enabled card games for incidental vocabulary acquisition in a foreign language

Ian McGraw, Brandon Yoshimoto, Stephanie Seneff
2009 Speech Communication  
We then turn to assessing the effects of the Word War game on vocabulary retention in a controlled environment.  ...  To assess long term learning gains as a function of time-on-task, we had the students interact with each system twice over a period of three weeks.  ...  Ming Zhu and James McGraw provided valuable discussions that contributed to the analysis of the studies presented in this work.  ... 
doi:10.1016/j.specom.2009.04.011 fatcat:z2g6w5nozbc6hadvomlogv6ej4

Insta-Reviewer: A Data-Driven Approach for Generating Instant Feedback on Students' Project Reports

Qinjin Jia, Mitchell Young, Yunkai Xiao, Jialin Cui, Chengyuan Liu, Parvez Rashid, Edward Gehringer, Antonija Mitrovic, Nigel Bosch
2022 Zenodo  
In this paper, we present a novel data-driven system, named Insta-Reviewer, for automatically generating instant feedback on students' project reports, using state-of-the-art natural language processing  ...  However, to the best of our knowledge, no previous study has investigated automated feedback generation on students' project reports.  ...  Model-based metrics: Model-based metrics use learned representations of words and sentences to compute semantic similarity between generated and reference texts.  ... 
doi:10.5281/zenodo.6853098 fatcat:qcm2caxxznberbu5pdt7waodye

Pretrained Language Models for Text Generation: A Survey [article]

Junyi Li, Tianyi Tang, Wayne Xin Zhao, Jian-Yun Nie, Ji-Rong Wen
2022 arXiv   pre-print
Text Generation aims to produce plausible and readable text in a human language from input data.  ...  an effective PLM to serve as the generation model; and 3) how to effectively optimize PLMs given the reference text and to ensure that the generated texts satisfy special text properties.  ...  The Bilingual Evaluation Understudy (BLEU) [140] is one of the first metrics used to compare the similarity of two sentences.  ... 
arXiv:2201.05273v4 fatcat:pnffabspsnbhvo44gbaorhxc3a

Unsupervised Classification of Student Dialogue Acts with Query-Likelihood Clustering

Aysu Ezen-Can, Kristy Elizabeth Boyer
2013 Educational Data Mining  
In natural language tutorial dialogue, student dialogue moves hold important information about knowledge and goals, and are therefore an integral part of providing adaptive tutoring.  ...  This framework combines automated natural language processing with clustering and a novel adaptation of an information retrieval technique.  ...  Any opinions, findings, conclusions, or recommendations expressed in this report are those of the participants, and do not necessarily represent the official views, opinions, or policy of the National  ... 
dblp:conf/edm/Ezen-CanB13 fatcat:3suuhjcbsrgnzecvozrw5xuyoi
« Previous Showing results 1 — 15 out of 21,382 results