257 Hits in 0.95 sec

Attention-Based Capsule Networks with Dynamic Routing for Relation Extraction [article]

Ningyu Zhang, Shumin Deng, Zhanlin Sun, Xi Chen, Wei Zhang, Huajun Chen
2018 arXiv   pre-print
A capsule is a group of neurons, whose activity vector represents the instantiation parameters of a specific type of entity. In this paper, we explore the capsule networks used for relation extraction in a multi-instance multi-label learning framework and propose a novel neural approach based on capsule networks with attention mechanisms. We evaluate our method with different benchmarks, and it is demonstrated that our method improves the precision of the predicted relations. Particularly, we
more » ... ow that capsule networks improve multiple entity pairs relation extraction.
arXiv:1812.11321v1 fatcat:q3okntkh5bbzlno346wffilxfq

Transfer Learning for Relation Extraction via Relation-Gated Adversarial Learning [article]

Ningyu Zhang, Shumin Deng, Zhanlin Sun, Jiaoyan Chen, Wei Zhang, Huajun Chen
2019 arXiv   pre-print
., 2017; Zhang et al., 2018b; Zeng et al., 2018; Qin et al., 2018; Zhang et al., 2019) .  ...  (Zhang et al., 2018a) proposed an adversarial nets-based partial domain adaptation method to identify the source samples in instance level.  ... 
arXiv:1908.08507v1 fatcat:3dicu6q6kvfxrex3pfr5rezvkm

ProteinKG65: A Knowledge Graph for Protein Science [article]

Siyuan Cheng, Xiaozhuan Liang, Zhen Bi, Ningyu Zhang, Huajun Chen
2022 arXiv   pre-print
Existing data-centric methods for protein science generally cannot sufficiently capture and leverage biology knowledge, which may be crucial for many protein tasks. To facilitate research in this field, we create ProteinKG65, a knowledge graph for protein science. Using gene ontology and Uniprot knowledge base as a basis, we transform and integrate various kinds of knowledge with aligned descriptions and protein sequences respectively to GO terms and protein entities. ProteinKG65 is mainly
more » ... ated to providing a specialized protein knowledge graph, bringing the knowledge of Gene Ontology to protein function and structure prediction. The current version contains about 614,099 entities, 5,620,437 triples (including 5,510,437 protein-go triplets and 110,000 GO-GO triplets). We also illustrate the potential applications of ProteinKG65 with a prototype. We hope our released knowledge graph can help promote studies in AI for science.
arXiv:2207.10080v1 fatcat:ajav76nlkng6hjnbg5r6wvnpf4

Logic-guided Semantic Representation Learning for Zero-Shot Relation Classification [article]

Juan Li, Ruoxu Wang, Ningyu Zhang, Wen Zhang, Fan Yang, Huajun Chen
2020 arXiv   pre-print
., 2015; Zhang et al., 2018; Zhang et al., 2019a; Yu et al., 2020; Wang et al., 2020) , and some joint models like (Zheng et al., 2017; Ye et al., 2020) .  ...  Inspired by (Zhang et al., 2019b) , we apply an simple but effective embeddingbased method to incorporate symbolic rules into semantic space and generate E rl .  ... 
arXiv:2010.16068v1 fatcat:32mls4yobvdr5lzyy3wd4o2ojq

Retrosplenial cortex and its role in spatial cognition [article]

Anna S Mitchell, Rafał Czajkowski, Ningyu Zhang, Kate Jeffery, Andrew Nelson
2017 bioRxiv   pre-print
Retrosplenial cortex (RSC) is a region within the posterior neocortical system, heavily interconnected with an array of brain networks, both cortical and subcortical, that is engaged by a myriad of cognitive tasks. Although there is no consensus as to its precise function, evidence from both human and animal studies clearly points to a role in spatial cognition. However, the spatial processing impairments that follow RSC damage are not straightforward to characterise, leading to difficulties in
more » ... defining the exact nature of its role. In the present article we review this literature and classify the types of ideas that have been put forward into three broad, somewhat overlapping classes: (i) Learning of landmark location, stability and permanence; (ii) Integration between spatial reference frames, and (iii) Consolidation and retrieval of spatial knowledge ('schemas'). We evaluate these models and suggest ways to test them, before briefly discussing whether the spatial function may be a subset of a more general function in episodic memory.
doi:10.1101/190801 fatcat:zhdtzdiljrbvxl5nklq4kr5mgm

PRGC: Potential Relation and Global Correspondence Based Joint Relational Triple Extraction [article]

Hengyi Zheng, Rui Wen, Xi Chen, Yifan Yang, Yunyan Zhang, Ziheng Zhang, Ningyu Zhang, Bin Qin, Ming Xu, Yefeng Zheng
2021 arXiv   pre-print
Joint extraction of entities and relations from unstructured texts is a crucial task in information extraction. Recent methods achieve considerable performance but still suffer from some inherent limitations, such as redundancy of relation prediction, poor generalization of span-based extraction and inefficiency. In this paper, we decompose this task into three subtasks, Relation Judgement, Entity Extraction and Subject-object Alignment from a novel perspective and then propose a joint
more » ... l triple extraction framework based on Potential Relation and Global Correspondence (PRGC). Specifically, we design a component to predict potential relations, which constrains the following entity extraction to the predicted relation subset rather than all relations; then a relation-specific sequence tagging component is applied to handle the overlapping problem between subjects and objects; finally, a global correspondence component is designed to align the subject and object into a triple with low-complexity. Extensive experiments show that PRGC achieves state-of-the-art performance on public benchmarks with higher efficiency and delivers consistent performance gain on complex scenarios of overlapping triples.
arXiv:2106.09895v1 fatcat:5uqfov5zdzhqbbmea4ja5yiupq

Kformer: Knowledge Injection in Transformer Feed-Forward Layers [article]

Yunzhi Yao, Shaohan Huang, Ningyu Zhang, Li Dong, Furu Wei, Huajun Chen
2022 arXiv   pre-print
Knowledge-Enhanced Model have developed a diverse set of techniques for knowledge integration on different knowledge sources. However, most previous work neglect the language model's own ability and simply concatenate external knowledge at the input. Recent work proposed that Feed Forward Network (FFN) in pre-trained language model can be seen as an memory that stored factual knowledge. In this work, we explore the FFN in Transformer and propose a novel knowledge fusion model, namely Kformer,
more » ... ich incorporates external knowledge through the feed-forward layer in Transformer. We empirically find that simply injecting knowledge into FFN can enhance the pre-trained language model's ability and facilitate current knowledge fusion methods. Our results on two benchmarks in the commonsense reasoning (i.e., SocialIQA) and medical question answering (i.e., MedQA-USMLE) domains demonstrate that Kformer can utilize external knowledge deeply and achieves absolute improvements in these tasks.
arXiv:2201.05742v1 fatcat:jsofnqtlfngxpdqx4jrzelm5eq

Conceptualized Representation Learning for Chinese Biomedical Text Mining [article]

Ningyu Zhang, Qianghuai Jia, Kangping Yin, Liang Dong, Feng Gao, Nengwei Hua
2020 arXiv   pre-print
Biomedical text mining is becoming increasingly important as the number of biomedical documents and web data rapidly grows. Recently, word representation models such as BERT has gained popularity among researchers. However, it is difficult to estimate their performance on datasets containing biomedical texts as the word distributions of general and biomedical corpora are quite different. Moreover, the medical domain has long-tail concepts and terminologies that are difficult to be learned via
more » ... nguage models. For the Chinese biomedical text, it is more difficult due to its complex structure and the variety of phrase combinations. In this paper, we investigate how the recently introduced pre-trained language model BERT can be adapted for Chinese biomedical corpora and propose a novel conceptualized representation learning approach. We also release a new Chinese Biomedical Language Understanding Evaluation benchmark (ChineseBLUE). We examine the effectiveness of Chinese pre-trained models: BERT, BERT-wwm, RoBERTa, and our approach. Experimental results on the benchmark show that our approach could bring significant gain. We release the pre-trained model on GitHub:
arXiv:2008.10813v1 fatcat:dqt4id37gjdalcqpxkvemkb7tm

Long-tail Relation Extraction via Knowledge Graph Embeddings and Graph Convolution Networks [article]

Ningyu Zhang, Shumin Deng, Zhanlin Sun, Guanying Wang, Xi Chen, Wei Zhang, Huajun Chen
2019 arXiv   pre-print
., 2016; Zhang et al., 2018a) .  ...  Relation structure (relational knowledge) has been studied and is quite effective for KG completion (Zhang et al., 2018b) .  ... 
arXiv:1903.01306v1 fatcat:jz3yzkhkhfcungwzkws4sytjke

Environment symmetry drives a multidirectional code in rat retrosplenial cortex [article]

Ningyu Zhang, Roddy M Grieves, Kate J Jeffery
2021 bioRxiv   pre-print
Author contributions: Ningyu Zhang: Conceptualization, Methodology, Software, Investigation, Formal analysis, Data curation, Visualization, Writing -Original Draft, Review & Editing; Roddy Grieves: Conceptualization  ... 
doi:10.1101/2021.08.22.457261 fatcat:mwg4u5j2engo5a44r3zjspev4y

LaKo: Knowledge-driven Visual Question Answering via Late Knowledge-to-Text Injection [article]

Zhuo Chen, Yufeng Huang, Jiaoyan Chen, Yuxia Geng, Yin Fang, Jeff Pan, Ningyu Zhang, Wen Zhang
2022 arXiv   pre-print
Visual question answering (VQA) often requires an understanding of visual concepts and language semantics, which relies on external knowledge. Most existing methods exploit pre-trained language models or/and unstructured text, but the knowledge in these resources are often incomplete and noisy. Some methods prefer to use knowledge graphs (KGs) which often have intensive structured knowledge, but the research is still quite preliminary. In this paper, we propose LaKo, a knowledge-driven VQA
more » ... d via Late Knowledge-to-text Injection. To effectively incorporate an external KG, we transfer triples into text and propose a late injection mechanism. Finally we address VQA as a text generation task with an effective encoder-decoder paradigm. In the evaluation with OKVQA datasets, our method achieves state-of-the-art results.
arXiv:2207.12888v1 fatcat:ru2vqlszxvb65laeokebt76itu

Bridging Text and Knowledge with Multi-Prototype Embedding for Few-Shot Relational Triple Extraction [article]

Haiyang Yu, Ningyu Zhang, Shumin Deng, Hongbin Ye, Wei Zhang, Huajun Chen
2020 arXiv   pre-print
Current supervised relational triple extraction approaches require huge amounts of labeled data and thus suffer from poor performance in few-shot settings. However, people can grasp new knowledge by learning a few instances. To this end, we take the first step to study the few-shot relational triple extraction, which has not been well understood. Unlike previous single-task few-shot problems, relational triple extraction is more challenging as the entities and relations have implicit
more » ... s. In this paper, We propose a novel multi-prototype embedding network model to jointly extract the composition of relational triples, namely, entity pairs and corresponding relations. To be specific, we design a hybrid prototypical learning mechanism that bridges text and knowledge concerning both entities and relations. Thus, implicit correlations between entities and relations are injected. Additionally, we propose a prototype-aware regularization to learn more representative prototypes. Experimental results demonstrate that the proposed method can improve the performance of the few-shot triple extraction.
arXiv:2010.16059v1 fatcat:ioxbuwd2qzgoxgux7fxmyfrpjy

Context-aware Deep Model for Entity Recommendation in Search Engine at Alibaba [article]

Qianghuai Jia, Ningyu Zhang, Nengwei Hua
2019 arXiv   pre-print
ACKNOWLEDGMENTS We would like to thank colleagues of our team -Xiangzhi Wang, Yulin Wang, Liang Dong, Kangping Yin, Zhenxin Ma, Yongjin Wang, Qiteng Yang, Wei Shen, Liansheng Sun, Kui Xiong, Weixing Zhang  ... 
arXiv:1909.04493v1 fatcat:eg7pou6l55ajpddguwp4n4lngi

Knowledge Extraction in Low-Resource Scenarios: Survey and Perspective [article]

Shumin Deng, Ningyu Zhang, Hui Chen, Feiyu Xiong, Jeff Z. Pan, Huajun Chen
2022 arXiv   pre-print
Preliminary on Low-resource KE 2.1 Knowledge Extraction Knowledge Extraction (KE) [Zhang et al., 2020a] aims to extract structural information from unstructured texts, such as Named Entity Recognition  ...  Meta Learning Qu et al., 2020] ; Shen et al., 2021] Transfer Learning [Huang et al., 2018; ; [Zhang et al., 2020b; Xue et al., 2020] ; [Soares et al., 2019; Wang et al., 2021b] Prompt Learning  ... 
arXiv:2202.08063v1 fatcat:2q64tx2mzne53gt24adi6ymj7a

Reasoning Through Memorization: Nearest Neighbor Knowledge Graph Embeddings [article]

Ningyu Zhang, Xin Xie, Xiang Chen, Shumin Deng, Chuanqi Tan, Fei Huang, Xu Cheng, Huajun Chen
2022 arXiv   pre-print
., 2015; Zhang et al., 2019a; Zhang et al., 2020; Zhang et al., 2021a; Qi et al., 2021; Zhang et al., 2021b] .  ...  objective of translation-based methods is that the translated head entity should be close to the tail entity in real space [Bordes et al., 2013], complex space [Sun et al., 2019], or quaternion space [Zhang  ... 
arXiv:2201.05575v1 fatcat:umhtd3wp75h2doegoe7q3yd3ve
« Previous Showing results 1 — 15 out of 257 results