Filters








11,652 Hits in 6.8 sec

Explicit Sparse Self-Attentive Network for CTR Prediction

Yu Luo, Wanwan Peng, Youping Fan, Hong Pang, Xiang Xu, Xiaohua Wu
2021 Procedia Computer Science  
In this paper, we propose a novel model ESAtInt to model high-dimensional and sparse features, while being able to explicitly select meaningful higher-order feature interactions and eliminate the impact  ...  In this paper, we propose a novel model ESAtInt to model high-dimensional and sparse features, while being able to explicitly select meaningful higher-order feature interactions and eliminate the impact  ...  Acknowledgement This work is supported by the grant of Sichuan Science and Technology Program 2018GZDZX0042 and 2018HH0061.  ... 
doi:10.1016/j.procs.2021.02.116 fatcat:7rdtzwqdfza3rmjwjyfkd65myy

AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks [article]

Weiping Song, Chence Shi, Zhiping Xiao, Zhijian Duan, Yewen Xu, Ming Zhang, Jian Tang
2018 arXiv   pre-print
Afterward, a multi-head self-attentive neural network with residual connections is proposed to explicitly model the feature interactions in the low-dimensional space.  ...  With different layers of the multi-head self-attentive neural networks, different orders of feature combinations of input features can be modeled.  ...  for the problem. • We propose a novel approach based on self-attentive neural network, which can automatically learn high-order feature interactions and efficiently handle large-scale highdimensional  ... 
arXiv:1810.11921v1 fatcat:wbxisocj75didheq546gaucnq4

DCAP: Deep Cross Attentional Product Network for User Response Prediction [article]

Zekai Chen, Fangtian Zhong, Zhumin Chen, Xiao Zhang, Robert Pless, Xiuzhen Cheng
2021 arXiv   pre-print
Prior studies in predicting user response leveraged the feature interactions by enhancing feature vectors with products of features to model second-order or high-order cross features, either explicitly  ...  This work aims to fill this gap by proposing a novel architecture Deep Cross Attentional Product Network (DCAP), which keeps cross network's benefits in modeling high-order feature interactions explicitly  ...  However, these models' performance can be limited in modeling high-order complicated feature interactions by merely stacking the self-attention blocks.  ... 
arXiv:2105.08649v2 fatcat:vjzva2ytlbc5jbpmxqjjwjvktm

Extended Factorization Machines for Sequential Recommendation

Nuan Wen, Fang Zhang
2020 IEEE Access  
Recently, a new surge of interest aims to use recurrent neural networks(RNN) to encode users' dynamic features with temporal characteristics.  ...  Furthermore, we merge extended-FM into higher-order interaction framework without significant changes to the deeper models themselves. We conduct comprehensive experiments on two real-world datasets.  ...  There is only a shallow component with the purpose of learning low-order feature interactions. Therefore, it is promising to exploit DNNs into FMs to learn high-order feature interactions.  ... 
doi:10.1109/access.2020.2977231 fatcat:s652n7nijfguzeq7lak53azczi

Sequence-Aware Factorization Machines for Temporal Predictive Analytics [article]

Tong Chen, Hongzhi Yin, Quoc Viet Hung Nguyen, Wen-Chih Peng, Xue Li, Xiaofang Zhou
2019 arXiv   pre-print
As a widely adopted solution, models based on Factorization Machines (FMs) are capable of modelling high-order interactions among features for effective sparse predictive analytics.  ...  As static features (e.g., user gender) and dynamic features (e.g., user interacted items) express different semantics, we innovatively devise a multi-view self-attention scheme that separately models the  ...  [15] , or fusing shallow low-order output with dense high-order output via Wide&Deep [18] , DeepFM [20] and xDeepFM [19] .  ... 
arXiv:1911.02752v2 fatcat:if242oj7pbbjdhyfnsp6ezwpxe

Sequence-Aware Factorization Machines for Temporal Predictive Analytics

Tong Chen, Hongzhi Yin, Quoc Viet Hung Nguyen, Wen-Chih Peng, Xue Li, Xiaofang Zhou
2020 2020 IEEE 36th International Conference on Data Engineering (ICDE)  
As a widely adopted solution, models based on Factorization Machines (FMs) are capable of modelling high-order interactions among features for effective sparse predictive analytics.  ...  As static features (e.g., user gender) and dynamic features (e.g., user interacted items) express different semantics, we innovatively devise a multi-view self-attention scheme that separately models the  ...  [15] , or fusing shallow low-order output with dense high-order output via Wide&Deep [18] , DeepFM [20] and xDeepFM [19] .  ... 
doi:10.1109/icde48307.2020.00125 dblp:conf/icde/ChenYNP0020 fatcat:k4s7gc3j2jggpk4gwwolmdwnxe

Disentangled Self-Attentive Neural Networks for Click-Through Rate Prediction [article]

Yichen Xu, Yanqiao Zhu, Feng Yu, Qiang Liu, Shu Wu
2021 arXiv   pre-print
To explicitly model high-order feature interaction, an efficient way is to perform inner product of feature embeddings with self-attentive neural networks.  ...  Due to the nature of data sparsity and high dimensionality in CTR prediction, a key to making effective prediction is to model high-order feature interaction among feature fields.  ...  To model arbitrary-order feature interaction, we can stack multiple layers of self-attentive networks with residual connections.  ... 
arXiv:2101.03654v2 fatcat:wth7jktdrnel3esk2bzu3pmxoe

AdnFM: An Attentive DenseNet based Factorization Machine for CTR Prediction [article]

Kai Wang, Chunxu Shen, Chaoyun Zhang Wenye Ma
2021 arXiv   pre-print
In this paper, we aim to explore more about these high-order features interactions. However, high-order feature interaction deserves more attention and further development.  ...  Factorization Machines and their variants consider pair-wise feature interactions, but normally we won't do high-order feature interactions using FM due to high time complexity.  ...  There are two major parts in Adn: 1), It uses a feed-forward neural network with hidden layers in the same size, which is treated as implicit high-order features. 2), It uses an attention mechanism to  ... 
arXiv:2012.10820v2 fatcat:d54opn7wlzhddkge7lc3haqby4

Generating Robust Audio Adversarial Examples with Temporal Dependency

Hongting Zhang, Pan Zhou, Qiben Yan, Xiao-Yang Liu
2020 Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence  
Following our observation, we leverage a proportional clipping strategy to reduce noise during the low-intensity periods.  ...  Experimental results and user study both suggest that the generated adversarial examples can significantly reduce human-perceptible noises and resist the defenses based on the temporal structure.  ...  part models the non-linear high-order feature interactions.  ... 
doi:10.24963/ijcai.2020/434 dblp:conf/ijcai/LuYCWLY20 fatcat:hwbh7z2gtna7vielahxoh42mxe

Usability and Security Testing of Online Links: A Framework for Click-Through Rate Prediction Using Deep Learning

Robertas Damaševičius, Ligita Zailskaitė-Jakštė
2022 Electronics  
We propose a Human-Centric Cyber Security (HCCS) model that additionally includes AI techniques targeted at the key elements of user, usage, and usability.  ...  The user, usage, and usability (3U's) are three principal constituents for cyber security.  ...  The FM unit can model both linear (1st order) and 2nd order interactions between features. The DNN unit is a feed-forward neural network (FFNN), which learns high-order feature combinations.  ... 
doi:10.3390/electronics11030400 fatcat:nmgvjbvpnbgk5kebmvp4xubrce

GraphFM: Graph Factorization Machines for Feature Interaction Modeling [article]

Zekun Li, Shu Wu, Zeyu Cui, Xiaoyu Zhang
2022 arXiv   pre-print
Factorization machine (FM) is a prevalent approach to modeling pairwise (second-order) feature interactions when dealing with high-dimensional sparse data.  ...  Then our proposed model which integrates the interaction function of FM into the feature aggregation strategy of Graph Neural Network (GNN), can model arbitrary-order feature interactions on the graph-structured  ...  It can be also combined with DNN which model implicit and explicit interactions simultaneously. • AutoInt [38] (D) uses self-attentive network to learn high-order feature interactions explicitly.  ... 
arXiv:2105.11866v3 fatcat:czqpjovrm5hdjcnifqhsfx3jmu

DexDeepFM: Ensemble Diversity Enhanced Extreme Deep Factorization Machine Model [article]

Ling Chen, Hongyu Shi
2021 arXiv   pre-print
In addition, the attention mechanism is introduced to discriminate the importance of ensemble diversity measures with different feature interaction orders.  ...  To identify predictive features from raw data, the state-of-the-art extreme deep factorization machine model (xDeepFM) introduces a new interaction network to leverage feature interactions at the vector-wise  ...  ., feature engineering or FM) and a deep part, are applied to capture both low-order and high-order feature interactions, e.g., Wide & Deep [8] and DeepFM [9] .  ... 
arXiv:2104.01924v2 fatcat:reynx72yjvaozjs4mgcksddyyy

SHORING: Design Provable Conditional High-Order Interaction Network via Symbolic Testing [article]

Hui Li, Xing Fu, Ruofan Wu, Jinyu Xu, Kai Xiao, Xiaofu Chang, Weiqiang Wang, Shuai Chen, Leilei Shi, Tao Xiong, Yuan Qi
2021 arXiv   pre-print
We argue that SHORING is capable of learning certain standard symbolic expressions which the standard multi-head self-attention network fails to learn, and conduct comprehensive experiments and ablation  ...  The event network learns arbitrarily yet efficiently high-order event-level embeddings via a provable reparameterization trick, the sequence network aggregates from sequence of event-level embeddings.  ...  For example, [44] uses FM to learn second-order feature interaction and uses a self-attention network to learn the importance of each event.  ... 
arXiv:2107.01326v1 fatcat:qw2ardgfsvgivlseyokkq56r4q

A Multi-Granular Aggregation-Enhanced Knowledge Graph Representation for Recommendation

Xi Liu, Rui Song, Yuhang Wang, Hao Xu
2022 Information  
nodes in the heterogeneous network into three categories—users, items, and entities, and connects the edges according to the similarity between the users and items so as to enhance the high-order connectivity  ...  In addition, we used attention mechanisms, the factorization machine (FM), and transformer (Trm) algorithms to aggregate messages from multi-granularity and different types to improve the representation  ...  Acknowledgments: The authors would like to thank all of anonymous reviewers and editors for their helpful suggestions for the improvement of this paper.  ... 
doi:10.3390/info13050229 fatcat:k4fqottd3vaoncy5r7kgdty4zi

Attention-based Multimodal Feature Representation Model for Micro-video Recommendation [article]

Mohan Hasama, Jing Li
2022 arXiv   pre-print
In recommender systems, models mostly use a combination of embedding layers and multilayer feedforward neural networks.  ...  The high-dimensional sparse original features are downscaled in the embedding layer and then fed into the fully connected network to obtain prediction results.  ...  Baselines FM [26] : Factorization Machine, which simulates first-order feature importance and second-order feature interactions.  ... 
arXiv:2205.08982v1 fatcat:qzlzxcradvehbkagxhwmvrycq4
« Previous Showing results 1 — 15 out of 11,652 results