Solving Historical Dictionary Codes with a Neural Language Model

Christopher Chu, Raphael Valenti, Kevin Knight
2020 Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)   unpublished
We solve difficult word-based substitution codes by constructing a decoding lattice and searching that lattice with a neural language model. We apply our method to a set of enciphered letters exchanged between US Army General James Wilkinson and agents of the Spanish Crown in the late 1700s and early 1800s, obtained from the US Library of Congress. We are able to decipher 75.1% of the cipher-word tokens correctly.
doi:10.18653/v1/2020.emnlp-main.471 fatcat:7m6op4bmbrc5bljltrprakf6ce