Internet Archive Scholar

Explainable Publication Year Prediction of Eighteenth Century Texts with the BERT Model

Iiro Rastas, Yann Ciarán Ryan, Iiro Tiihonen, Mohammadreza Qaraei, Liina Repo, Rohit Babbar, Eetu Mäkelä, Mikko Tolonen, Filip Ginter
2022 Proceedings of the 3rd Workshop on Computational Approaches to Historical Language Change   unpublished

Preserved Fulltext

fulltext thumbnail
Web Archive Capture PDF (274.4 kB)
https://web.archive.org/web/20220516092308/https://aclanthology.org/2022.lchange-1.7.pdf

A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL. The file type is application/pdf.

In this paper, we describe a BERT model trained on the Eighteenth Century Collections Online (ECCO) dataset of digitized documents. The ECCO dataset poses unique modelling challenges due to the presence of Optical Character Recognition (OCR) artifacts. We establish the performance of the BERT model on a publication year prediction task against linear baseline models and human judgement, finding the BERT model to be superior to both and able to date the works, on average, with less than 7 years
more » ... bsolute error. We also explore how language change over time affects the model by analyzing the features the model uses for publication year predictions as given by the Integrated Gradients model explanation method.
doi:10.18653/v1/2022.lchange-1.7 fatcat:vmebyeb3zzajlcvj3fapf2jwpi
Citation

Iiro Rastas, Yann Ciarán Ryan, Iiro Tiihonen, Mohammadreza Qaraei, Liina Repo, Rohit Babbar, Eetu Mäkelä, Mikko Tolonen, Filip Ginter. "Explainable Publication Year Prediction of Eighteenth Century Texts with the BERT Model." Proceedings of the 3rd Workshop on Computational Approaches to Historical Language Change (2022)