Filters








3,704 Hits in 5.1 sec

Multi-Grained Knowledge Distillation for Named Entity Recognition

Xuan Zhou, Xiao Zhang, Chenyang Tao, Junya Chen, Bing Xu, Wei Wang, Jing Xiao
2021 Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies   unpublished
For many applications, including named entity recognition (NER), matching the state-of-the-art result under budget has attracted considerable attention.  ...  Drawing power from the recent advance in knowledge distillation (KD), this work presents a novel distillation scheme to efficiently transfer the knowledge learned from big models to their more affordable  ...  Acknowledgements The authors would like to thank the anonymous reviewers for their insightful comments. The research conducted at Duke was supported in part by the DOE, NSF and ONR.  ... 
doi:10.18653/v1/2021.naacl-main.454 fatcat:hn5n6gzw5jalnbcnk6ln4wr334

Distilling Knowledge from Well-Informed Soft Labels for Neural Relation Extraction

Zhenyu Zhang, Xiaobo Shu, Bowen Yu, Tingwen Liu, Jiapeng Zhao, Quangang Li, Li Guo
2020 PROCEEDINGS OF THE THIRTIETH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE AND THE TWENTY-EIGHTH INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE  
Furthermore, this model is regarded as teacher to generate well-informed soft labels and guide the optimization of a student network via knowledge distillation.  ...  Besides, a multi-aspect attention mechanism is introduced to help student mine latent information from text.  ...  Besides, named entity recognition (NER) tag is also a very useful shallow grammatical information, which can be treated as coarse-grained entity type.  ... 
doi:10.1609/aaai.v34i05.6509 fatcat:cqcm3hstd5gztbeyuwbo2rgqvy

Incorporating Explicit Knowledge in Pre-trained Language Models for Passage Re-ranking [article]

Qian Dong, Yiding Liu, Suqi Cheng, Shuaiqiang Wang, Zhicong Cheng, Shuzi Niu, Dawei Yin
2022 arXiv   pre-print
Besides, a novel knowledge injector is designed for the dynamic interaction between text and knowledge encoder.  ...  To leverage a reliable knowledge, we propose a novel knowledge graph distillation method and obtain a knowledge meta graph as the bridge between query and passage.  ...  , namely knowledge meta graph.  ... 
arXiv:2204.11673v1 fatcat:5iynilhkhbdqnogc3flfylt4wu

Incorporating Explicit Knowledge in Pre-trained Language Models for Passage Re-ranking

Qian Dong, Yiding Liu, Suqi Cheng, Shuaiqiang Wang, Zhicong Cheng, Shuzi Niu, Dawei Yin
2022 Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval  
Besides, a novel knowledge injector is designed for the dynamic interaction between text and knowledge encoder.  ...  To leverage a reliable knowledge, we propose a novel knowledge graph distillation method and obtain a knowledge meta graph as the bridge between query and passage.  ...  , namely knowledge meta graph.  ... 
doi:10.1145/3477495.3531997 fatcat:uzhzesb3cvdkhi4cszebvsg5ki

ScienceExamCER: A High-Density Fine-Grained Science-Domain Corpus for Common Entity Recognition [article]

Hannah Smith, Zeyu Zhang, John Culnan, Peter Jansen
2019 arXiv   pre-print
We show an off-the-shelf BERT-based named entity recognition model modified for multi-label classification achieves an accuracy of 0.85 F1 on this task, suggesting strong utility for downstream tasks in  ...  Named entity recognition identifies common classes of entities in text, but these entity labels are generally sparse, limiting utility to downstream tasks.  ...  Named entity recognition identifies common classes of entities in text, but these entity labels are generally sparse (typically occurring for between 10% to 20% of words in a corpus, see Section 3.4.),  ... 
arXiv:1911.10436v1 fatcat:2yasc52vvza5tfksqgdmxizrsy

Fine-grained Knowledge Fusion for Sequence Labeling Domain Adaptation [article]

Huiyun Yang, Shujian Huang, Xinyu Dai, Jiajun Chen
2019 arXiv   pre-print
To take the multi-level domain relevance discrepancy into account, in this paper, we propose a fine-grained knowledge fusion model with the domain relevance modeling scheme to control the balance between  ...  Experiments on three sequence labeling tasks show that our fine-grained knowledge fusion model outperforms strong baselines and other state-of-the-art sequence labeling domain adaptation methods.  ...  Acknowledgements We would like to thank the anonymous reviewers for their insightful comments. Shujian Huang is the corresponding author.  ... 
arXiv:1909.04315v1 fatcat:qdkz4x4u75h4vbwkh5hx57kina

A Fresh Look on Knowledge Bases

Erdal Kuzey, Jilles Vreeken, Gerhard Weikum
2014 Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management - CIKM '14  
However, their knowledge of named events like sports finals, political scandals, or natural disasters is fairly limited, as these are continuously emerging entities.  ...  This paper presents a method for extracting named events from news articles, reconciling them into canonicalized representation, and organizing them into finegrained semantic classes to populate a knowledge  ...  It is worth noting that the entity set of a node in G can be noisy due to imperfect quality of the named entity recognition tool.  ... 
doi:10.1145/2661829.2661984 dblp:conf/cikm/KuzeyVW14 fatcat:hgm6kv62kjgdlidyeh4shcgrfa

Learning from Noisy Labels with Distillation [article]

Yuncheng Li, Jianchao Yang, Yale Song, Liangliang Cao, Jiebo Luo, Li-Jia Li
2017 arXiv   pre-print
In this work, we propose a unified distillation framework to use side information, including a small clean dataset and label relations in knowledge graph, to "hedge the risk" of learning from noisy labels  ...  The ability of learning from noisy labels is very useful in many visual recognition tasks, as a vast amount of data with noisy labels are relatively easy to obtain.  ...  As shown in the analysis of Section 3.2, the distillation parameters λ and knowledge graph G need to be properly chosen and designed, in order for the soft labels to achieve Name Clean Set D c Noisy Set  ... 
arXiv:1703.02391v2 fatcat:2wctk4ub7jabzhf32njltr3aiy

Knowledge Extraction in Low-Resource Scenarios: Survey and Perspective [article]

Shumin Deng, Ningyu Zhang, Hui Chen, Feiyu Xiong, Jeff Z. Pan, Huajun Chen
2022 arXiv   pre-print
In addition, we describe promising applications and outline some potential directions for future research.  ...  Knowledge Extraction (KE) which aims to extract structural information from unstructured texts often suffers from data scarcity and emerging unseen types, i.e., low-resource scenarios.  ...  For instance, given a sentence "Jack is married to the Iraqi microbiologist known as Dr. Germ.": Named Entity Recognition should identify the types of entities, e.g., 'Jack', 'Dr.  ... 
arXiv:2202.08063v1 fatcat:2q64tx2mzne53gt24adi6ymj7a

Fine-grained Knowledge Fusion for Sequence Labeling Domain Adaptation

Huiyun Yang, Shujian Huang, XIN-YU DAI, Jiajun CHEN
2019 Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)  
To take the multi-level domain relevance discrepancy into account, in this paper, we propose a fine-grained knowledge fusion model with the domain relevance modeling scheme to control the balance between  ...  Experiments on three sequence labeling tasks show that our fine-grained knowledge fusion model outperforms strong baselines and other stateof-the-art sequence labeling domain adaptation methods. 1  ...  Acknowledgements We would like to thank the anonymous reviewers for their insightful comments. Shujian Huang is the corresponding author. This work is supported by National Science Foundation of China  ... 
doi:10.18653/v1/d19-1429 dblp:conf/emnlp/YangHDC19 fatcat:ozkh4iwknfdbnk4zmwwpxssmri

Fine-grainedWeb Content Classificationvia Entity-level Analytics:The Case ofSemantic Fingerprinting

Céline Alec, Marc Spaniol
2019 Journal of Web Engineering  
However, the whole task is not as simple as classifying something as A or B, but to assign the most suitable (sub-)category for each Web content based on a fine-grained classification scheme.  ...  Having such a fine-grained type hierarchy, the engineering task of Web content classification becomes out-most challenging. Our main objective in this work is to investigate whether  ...  We thank our colleagues for the inspiring discussions.  ... 
doi:10.13052/jwe1540-9589.17673 fatcat:zdn43vkaljbxhcskl23x3zftw4

Cross-Domain NER using Cross-Domain Language Modeling

Chen Jia, Xiaobo Liang, Yue Zhang
2019 Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics  
Due to limitation of labeled resources, crossdomain named entity recognition (NER) has been a challenging task.  ...  To address this issue, we consider using cross-domain LM as a bridge cross-domains for NER domain adaptation, performing crossdomain and cross-task knowledge transfer by designing a novel parameter generation  ...  We also thank the anonymous reviewers for their helpful comments and suggestions.  ... 
doi:10.18653/v1/p19-1236 dblp:conf/acl/JiaXZ19 fatcat:zanegctwk5aflemccs7apfut44

EasyNLP: A Comprehensive and Easy-to-use Toolkit for Natural Language Processing [article]

Chengyu Wang, Minghui Qiu, Taolin Zhang, Tingting Liu, Lei Li, Jianing Wang, Ming Wang, Jun Huang, Wei Lin
2022 arXiv   pre-print
It further features knowledge-enhanced pre-training, knowledge distillation and few-shot learning functionalities for large-scale PTMs, and provides a unified framework of model training, inference and  ...  deployment for real-world applications.  ...  Acknowledgments We thank Haojie Pan, Peng Li, Boyu Hou, Xiaoqing Chen, Xiaodan Wang, Xiangru Zhu and many other members of the Alibaba PAI team for their contribution and suggestions on building the EasyNLP  ... 
arXiv:2205.00258v1 fatcat:7bi6t6ymnjhnfdiyejswu35dki

Few-Shot Cross-lingual Transfer for Coarse-grained De-identification of Code-Mixed Clinical Texts [article]

Saadullah Amin, Noon Pokaratsiri Goldstein, Morgan Kelly Wixted, Alejandro García-Rudolph, Catalina Martínez-Costa, Günter Neumann
2022 arXiv   pre-print
In this work, we empirically show the few-shot cross-lingual transfer property of LMs for named entity recognition (NER) and apply it to solve a low-resource and real-world challenge of code-mixed (Spanish-Catalan  ...  Despite the advances in digital healthcare systems offering curated structured knowledge, much of the critical information still lies in large volumes of unlabeled and unstructured clinical texts.  ...  Acknowledgments The authors would like to thank the anonymous reviewers and Josef van Genabith for their helpful feedback.  ... 
arXiv:2204.04775v1 fatcat:ai5t6ulki5anpm3kp4vxyt64he

Mengzi: Towards Lightweight yet Ingenious Pre-trained Models for Chinese [article]

Zhuosheng Zhang, Hanqing Zhang, Keming Chen, Yuhang Guo, Jingyun Hua, Yulong Wang, Ming Zhou
2021 arXiv   pre-print
This calls for the study of training more efficient models with less computation but still ensures impressive performance.  ...  This technical report releases our pre-trained model called Mengzi, which stands for a family of discriminative, generative, domain-specific, and multimodal pre-trained model variants, capable of a wide  ...  We extract the entities (e.g., events) from LUGE for the entity recognition task. 9 For evaluation on the other tasks, we use our self-collected datasets.  ... 
arXiv:2110.06696v2 fatcat:uwbje6ftcbbzhltkvnhsox6osu
« Previous Showing results 1 — 15 out of 3,704 results