Filters








4 Hits in 4.5 sec

What Changes Can Large-scale Language Models Bring? Intensive Study on HyperCLOVA: Billions-scale Korean Generative Pretrained Transformers [article]

Boseop Kim, HyoungSeok Kim, Sang-Woo Lee, Gichang Lee, Donghyun Kwak, Dong Hyeon Jeon, Sunghyun Park, Sungju Kim, Seonhoon Kim, Dongpil Seo, Heungsub Lee, Minyoung Jeong (+25 others)
2021 arXiv   pre-print
GPT-3 shows remarkable in-context learning ability of large-scale language models (LMs) trained on hundreds of billion scale data.  ...  To achieve this, we introduce HyperCLOVA, a Korean variant of 82B GPT-3 trained on a Korean-centric corpus of 560B tokens.  ...  In addition, the authors thank NAVER Cloud for technically supporting training environments of HyperCLOVA. Finally, the authors thank Reinald Kim Amplayo and Hwaran Lee for proofreading.  ... 
arXiv:2109.04650v2 fatcat:bervoiacjfbitm245s6iyw67xu

What Changes Can Large-scale Language Models Bring? Intensive Study on HyperCLOVA: Billions-scale Korean Generative Pretrained Transformers

Boseop Kim, HyoungSeok Kim, Sang-Woo Lee, Gichang Lee, Donghyun Kwak, Jeon Dong Hyeon, Sunghyun Park, Sungju Kim, Seonhoon Kim, Dongpil Seo, Heungsub Lee, Minyoung Jeong (+25 others)
2021 Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing   unpublished
GPT-3 shows remarkable in-context learning ability of large-scale language models (LMs) trained on hundreds of billion scale data.  ...  Enhanced by our Korean-specific tokenization, HyperCLOVA with our training configuration shows state-of-the-art in-context zero-shot and few-shot learning performances on various downstream tasks in Korean  ...  In addition, the authors thank NAVER Cloud for technically supporting training environments of HyperCLOVA. Finally, the authors thank Reinald Kim Amplayo and Hwaran Lee for proofreading.  ... 
doi:10.18653/v1/2021.emnlp-main.274 fatcat:h3he22rn4jbrrpkaqxzqtteztm

Transformer-based Korean Pretrained Language Models: A Survey on Three Years of Progress [article]

Kichang Yang
2021 arXiv   pre-print
Furthermore, in the past three years, various Pretrained Language Models specialized for Korean have appeared.  ...  , various methodologies, data, and models for learning the Pretrained Language Model began to appear.  ...  ., “What changes 1–9. can large-scale language models bring? intensive study [14] D. So, Q. Le, and C.  ... 
arXiv:2112.03014v1 fatcat:65ilb7fuijcmzb6coxaeqdsnp4

A Survey on Awesome Korean NLP Datasets [article]

Byunghyun Ban
2021 arXiv   pre-print
Although benchmark tests with English datasets are sufficient to show off the performances of new models and methods, still a researcher need to train and validate the models on Korean based datasets to  ...  produce a technology or product, suitable for Korean processing.  ...  "What changes can large-scale language models Bring? Intensive Study on HyperCLOVA: Billions-scale Korean bring?  ... 
arXiv:2112.01624v2 fatcat:xkq767m67nfypgz4nehle7pgci