143 Hits in 9.3 sec

Semantic Models for the First-stage Retrieval: A Comprehensive Review [article]

Yinqiong Cai, Yixing Fan, Jiafeng Guo, Fei Sun, Ruqing Zhang, Xueqi Cheng
2021 arXiv   pre-print
Therefore, it has been a long-term desire to build semantic models for the first-stage retrieval that can achieve high recall efficiently.  ...  In this paper, we describe the current landscape of the first-stage retrieval models under a unified framework to clarify the connection between classical term-based retrieval methods, early semantic retrieval  ...  For example, Dai and Callan [34, 36] proposed a BERT-based framework (DeepCT) to evaluate the term importance of sentences/passages in a context-aware manner.  ... 
arXiv:2103.04831v3 fatcat:6qa7hvc3jve3pcmo2mo4qsiefq


Nurendra Choudhary, Nikhil Rao, Sumeet Katariya, Karthik Subbian, Chandan K. Reddy
2022 Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining  
To address these challenges, in this paper, we propose an AtteNTive Hyperbolic Entity Model (ANTHEM), a novel attention-based product search framework that models query entities as two-vector hyperboloids  ...  Product search is a fundamentally challenging problem due to the large-size of product catalogues and the complexity of extracting semantic information from products.  ...  C QUERY-MATCHING ALGORITHM Algorithm 2 provides the pseudo-code for training ANTHEM for the task of query matching.  ... 
doi:10.1145/3488560.3498456 fatcat:5skmjm7x6vfeppjo7x5kamo4em

Pseudo Relevance Feedback with Deep Language Models and Dense Retrievers: Successes and Pitfalls [article]

Hang Li and Ahmed Mourad and Shengyao Zhuang and Bevan Koopman and Guido Zuccon
2022 arXiv   pre-print
Pseudo Relevance Feedback (PRF) is known to improve the effectiveness of bag-of-words retrievers.  ...  In this article, we address this gap by investigating methods for integrating PRF signals into rerankers and dense retrievers based on deep language models.  ...  For semantic matching, they split the top-𝑘 PRF documents into sentences. For each sentence, they use BERT to estimate the semantic similarity with the query.  ... 
arXiv:2108.11044v2 fatcat:mc2cgszfsbbnpfvwjtkeclbyw4

Neural Ranking Models for Document Retrieval [article]

Mohamed Trabelsi, Zhiyu Chen, Brian D. Davison, Jeff Heflin
2021 arXiv   pre-print
A variety of deep learning models have been proposed, and each model presents a set of neural network components to extract features that are used for ranking.  ...  These models are trained end-to-end to extract features from the raw data for ranking tasks, so that they overcome the limitations of hand-crafted features.  ...  A regression framework for learning ranking functions using relative relevance judgments.  ... 
arXiv:2102.11903v1 fatcat:zc2otf456rc2hj6b6wpcaaslsa

Pre-training Methods in Information Retrieval [article]

Yixing Fan, Xiaohui Xie, Yinqiong Cai, Jia Chen, Xinyu Ma, Xiangsheng Li, Ruqing Zhang, Jiafeng Guo
2022 arXiv   pre-print
The core of information retrieval (IR) is to identify relevant information from large-scale resources and return it as a ranked list to respond to the user's information need.  ...  In addition, we also introduce PTMs specifically designed for IR, and summarize available datasets as well as benchmark leaderboards.  ...  Naseri et al. (2021)to expand the original query or incorporate them with the effective pseudo feedback-based relevance model.To combine BERT embeddings with probabilistic language models,Naseri et al.  ... 
arXiv:2111.13853v3 fatcat:pilemnpphrgv5ksaktvctqdi4y

Pretrained Transformers for Text Ranking: BERT and Beyond [article]

Jimmy Lin, Rodrigo Nogueira, Andrew Yates
2021 arXiv   pre-print
We cover a wide range of modern techniques, grouped into two high-level categories: transformer models that perform reranking in multi-stage architectures and dense retrieval techniques that perform ranking  ...  There are two themes that pervade our survey: techniques for handling long documents, beyond typical sentence-by-sentence processing in NLP, and techniques for addressing the tradeoff between effectiveness  ...  Special thanks goes out to two anonymous reviewers for their insightful comments and helpful feedback.  ... 
arXiv:2010.06467v3 fatcat:obla6reejzemvlqhvgvj77fgoy

Pretrained Transformers for Text Ranking: BERT and Beyond

Andrew Yates, Rodrigo Nogueira, Jimmy Lin
2021 Proceedings of the 14th ACM International Conference on Web Search and Data Mining  
The goal of text ranking is to generate an ordered list of texts retrieved from a corpus in response to a query for a particular task.  ...  This survey provides an overview of text ranking with neural network architectures known as transformers, of which BERT is the best-known example.  ...  We'd like to thank the following people for comments on earlier drafts of this work: Maura Grossman, Sebastian Hofstätter, Xueguang Ma, and Bhaskar Mitra.  ... 
doi:10.1145/3437963.3441667 fatcat:6teqmlndtrgfvk5mneq5l7ecvq

Learning from Explanations with Neural Execution Tree [article]

Ziqi Wang, Yujia Qin, Wenxuan Zhou, Jun Yan, Qinyuan Ye, Leonardo Neves, Zhiyuan Liu, Xiang Ren
2020 arXiv   pre-print
In this paper, we propose a novel Neural Execution Tree (NExT) framework to augment training data for text classification using NL explanations.  ...  After transforming NL explanations into executable logical forms by semantic parsing, NExT generalizes different types of actions specified by the logical forms for labeling data instances, which substantially  ...  We would like to thank all the collaborators in USC INK research lab for their constructive feedback on the work.  ... 
arXiv:1911.01352v3 fatcat:fqbntowaj5addj4h3o6fasbykm

Extracting Prominent Aspects of Online Customer Reviews: A Data-Driven Approach to Big Data Analytics

Noaman M. Ali, Abdullah Alshahrani, Ahmed M. Alghamdi, Boris Novikov
2022 Electronics  
Extracting aspect terms for structure-free text is the primary task incorporated in the aspect-based sentiment analysis.  ...  Shufeng Xiong and Donghong Ji [22] proposed a semantically relevance-based model for extracting aspect-phrase. The authors have introduced a pipeline model of word embedding and clustering.  ...  Since our original dataset consists of the customers' reviews, each review could be divided into sentences, "list of sentences", and each sentence would be a list of tokens.  ... 
doi:10.3390/electronics11132042 fatcat:d3rg4gmzgrd7fcurzr3pe62ylq

A Survey of Quantum Theory Inspired Approaches to Information Retrieval [article]

Sagar Uprety and Dimitris Gkoumas and Dawei Song
2020 arXiv   pre-print
Since 2004, researchers have been using the mathematical framework of Quantum Theory (QT) in Information Retrieval (IR). QT offers a generalized probability and logic framework.  ...  Such a framework has been shown capable of unifying the representation, ranking and user cognitive aspects of IR, and helpful in developing more dynamic, adaptive and context-aware IR systems.  ...  The authors are hopeful that this literature survey is able to provide a clear picture of the quantum-inspired IR field and set a road-map for researchers to take this field forward.  ... 
arXiv:2007.04357v1 fatcat:uc6mi6mlizfivk2v3reb474s4q

Geometry-Aware Supertagging with Heterogeneous Dynamic Convolutions [article]

Konstantinos Kogkalidis, Michael Moortgat
2022 arXiv   pre-print
In this work, we revisit constructive supertagging from a graph-theoretic perspective, and propose a framework based on heterogeneous dynamic graph convolutions aimed at exploiting the distinctive structure  ...  of a supertagger's output space.  ...  To adapt their representation to our framework, we cast unary operators into pseudo-binaries by inserting an artificial terminal tree in a fixed slot within them.  ... 
arXiv:2203.12235v2 fatcat:jjpzb3owl5gtxdadkz2c6i556i

A survey of joint intent detection and slot filling models in natural language understanding

Henry Weld, Xiaoqi Huang, Siqu Long, Josiah Poon, Soyeon Caren Han
2022 ACM Computing Surveys  
Intent classification, to identify the speaker's intention, and slot filling, to label each token with a semantic type, are critical tasks in natural language understanding.  ...  More recently joint models, that address the two tasks together, have achieved state-of-the-art performance for each task, and have shown there exists a strong relationship between the two.  ...  A CNN encodes the representation into a vector, while a separate CNN encodes the sentence itself into another vector.  ... 
doi:10.1145/3547138 fatcat:sbv2bqasqba6zkojqmi4jb4blm

A Roadmap for Big Model [article]

Sha Yuan, Hanyu Zhao, Shuai Zhao, Jiahong Leng, Yangxiao Liang, Xiaozhi Wang, Jifan Yu, Xin Lv, Zhou Shao, Jiaao He, Yankai Lin, Xu Han (+88 others)
2022 arXiv   pre-print
With the rapid development of deep learning, training Big Models (BMs) for multiple downstream tasks becomes a popular paradigm.  ...  In this paper, we cover not only the BM technologies themselves but also the prerequisites for BM training and applications with BMs, dividing the BM review into four parts: Resource, Models, Key Technologies  ...  Formally, given a system output x and a reference x, BLEURT firstly maps the sentence pair (x, x) into a continuous vector using BERT: v BERT = BERT(x, x).  ... 
arXiv:2203.14101v4 fatcat:rdikzudoezak5b36cf6hhne5u4

Proceedings of the First Workshop on Weakly Supervised Learning (WeaSuL) [article]

Michael A. Hedderich, Benjamin Roth, Katharina Kann, Barbara Plank, Alex Ratner, Dietrich Klakow
2021 arXiv   pre-print
for prediction.  ...  In this workshop, we want to advance theory, methods and tools for allowing experts to express prior coded knowledge for automatic data annotations that can be used to train arbitrary deep neural networks  ...  We proposed a new framework based on total correlation for weakly-supervised disentanglement and showed through empirical evaluations on image datasets that our model improves learning disentangled representations  ... 
arXiv:2107.03690v1 fatcat:e57s4gr4lrdp5jtw7uoqsbtknq

A Review of Multi-Modal Learning from the Text-Guided Visual Processing Viewpoint

Ubaid Ullah, Jeong-Sik Lee, Chang-Hyeon An, Hyeonjin Lee, Su-Yeong Park, Rock-Hyun Baek, Hyun-Chul Choi
2022 Sensors  
For decades, co-relating different data domains to attain the maximum potential of machines has driven research, especially in neural networks.  ...  We broadly categorize text-guided visual output into three main divisions and meaningful subdivisions by critically examining an extensive body of literature from top-tier computer vision venues and closely  ...  framework with probabilistic modeling.  ... 
doi:10.3390/s22186816 pmid:36146161 pmcid:PMC9503702 fatcat:mqhcrujj5bbebgo2brdnad3p6m
« Previous Showing results 1 — 15 out of 143 results