Filters








2 Hits in 2.8 sec

UniDrop: A Simple yet Effective Technique to Improve Transformer without Extra Cost [article]

Zhen Wu, Lijun Wu, Qi Meng, Yingce Xia, Shufang Xie, Tao Qin, Xinyu Dai, Tie-Yan Liu
2021 arXiv   pre-print
With some explorations, we find simple techniques such as dropout, can greatly boost model performance with a careful design.  ...  Specifically, we propose an approach named UniDrop to unites three different dropout techniques from fine-grain to coarse-grain, i.e., feature dropout, structure dropout, and data dropout.  ...  Acknowledgments The authors would like to thank the anonymous reviewers for their valuable comments. Xinyu Dai and Lijun Wu are the corresponding authors.  ... 
arXiv:2104.04946v1 fatcat:6dqmk6fzbnh3fmvbee3gq3xkne

UniDrop: A Simple yet Effective Technique to Improve Transformer without Extra Cost

Zhen Wu, Lijun Wu, Qi Meng, Yingce Xia, Shufang Xie, Tao Qin, Xinyu Dai, Tie-Yan Liu
2021 Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies   unpublished
With some explorations, we find simple techniques such as dropout, can greatly boost model performance with a careful design.  ...  Specifically, we propose an approach named UniDrop to unite three different dropout techniques from fine-grain to coarse-grain, i.e., feature dropout, structure dropout, and data dropout.  ...  Acknowledgments The authors would like to thank the anonymous reviewers for their valuable comments. Xinyu Dai and Lijun Wu are the corresponding authors.  ... 
doi:10.18653/v1/2021.naacl-main.302 fatcat:oafhegiwrvbz7l5plzslvkihh4