Transparent text quality assessment with convolutional neural networks

Robert Östling, Gintare Grigonyte
2017 Proceedings of the 12th Workshop on Innovative Use of NLP for Building Educational Applications  
We present a very simple model for text quality assessment based on a deep convolutional neural network, where the only supervision required is one corpus of usergenerated text of varying quality, and one contrasting text corpus of consistently high quality. Our model is able to provide local quality assessments in different parts of a text, which allows visual feedback about where potentially problematic parts of the text are located, as well as a way to evaluate which textual features are
more » ... ured by our model. We evaluate our method on two corpora: a large corpus of manually graded student essays and a longitudinal corpus of language learner written production, and find that the text quality metric learned by our model is a fairly strong predictor of both essay grade and learner proficiency level.
doi:10.18653/v1/w17-5031 dblp:conf/bea/OstlingG17 fatcat:3pyjfyhvizbfbf32dqjy4g6qra