A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf
.
Inducing Transformer's Compositional Generalization Ability via Auxiliary Sequence Prediction Tasks
2021
Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
unpublished
Systematic compositionality is an essential mechanism in human language, allowing the recombination of known parts to create novel expressions. However, existing neural models have been shown to lack this basic ability in learning symbolic structures. Motivated by the failure of a Transformer model on the SCAN compositionality challenge (Lake and , which requires parsing a command into actions, we propose two auxiliary sequence prediction tasks as additional training supervision. These
doi:10.18653/v1/2021.emnlp-main.505
fatcat:4xm6czoc6zbpvdtclpesv73zw4