On Difficulties of Cross-Lingual Transfer with Order Differences: A Case Study on Dependency Parsing [article]

Wasi Uddin Ahmad, Zhisong Zhang, Xuezhe Ma, Eduard Hovy, Kai-Wei Chang, Nanyun Peng
<span title="2019-04-17">2019</span> <i > arXiv </i> &nbsp; <span class="release-stage" >pre-print</span>
Different languages might have different word orders. In this paper, we investigate cross-lingual transfer and posit that an order-agnostic model will perform better when transferring to distant foreign languages. To test our hypothesis, we train dependency parsers on an English corpus and evaluate their transfer performance on 30 other languages. Specifically, we compare encoders and decoders based on Recurrent Neural Networks (RNNs) and modified self-attentive architectures. The former relies
more &raquo; ... on sequential information while the latter is more flexible at modeling word order. Rigorous experiments and detailed analysis shows that RNN-based architectures transfer well to languages that are close to English, while self-attentive models have better overall cross-lingual transferability and perform especially well on distant languages.
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/1811.00570v3">arXiv:1811.00570v3</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/n47ecgxpxbcitaply4ykgpns4m">fatcat:n47ecgxpxbcitaply4ykgpns4m</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20200930154228/https://arxiv.org/pdf/1811.00570v3.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/55/71/55711716d36f17af722c9d38dc8dec109b2e70a6.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/1811.00570v3" title="arxiv.org access"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> arxiv.org </button> </a>