A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Filters
Mathematical Reasoning via Self-supervised Skip-tree Training
[article]
2020
arXiv
pre-print
Preserved Fulltext
Other Versions
We examine whether self-supervised language modeling applied to mathematical formulas enables logical reasoning. ...
We find that models trained on the skip-tree task show surprisingly strong mathematical reasoning abilities, and outperform models trained on standard skip-sequence tasks. ...
Self-supervised training techniques for formal mathematics have received much less attention. ...
arXiv:2006.04757v3
fatcat:yrmqpmijjzh6rcnkm3b77k3mwy
Towards the Automatic Mathematician
[chapter]
2021
Lecture Notes in Computer Science
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
AbstractOver the recent years deep learning has found successful applications in mathematical reasoning. ...
This extended abstract summarizes recent developments of machine learning in mathematical reasoning and the vision of the N2Formal group at Google Research to create an automatic mathematician. ...
(skip-tree training) [41] . ...
doi:10.1007/978-3-030-79876-5_2
fatcat:w6hmshg5gzbabdfkyj5om3yeme
Proof Artifact Co-training for Theorem Proving with Language Models
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Other Versions
We propose PACT (Proof Artifact Co-Training), a general methodology for extracting abundant self-supervised data from kernel-level proof terms for co-training alongside the usual tactic prediction objective ...
We apply this methodology to Lean, an interactive proof assistant which hosts some of the most sophisticated formalized mathematics to date. ...
Unlike skip-tree training , which focuses solely on predicting masked subterms of theorem statements, PACT derives its self-supervised training data from far more complex proofs. ...
arXiv:2102.06203v2
fatcat:ly2rlwm2erhjhakw42d2nht2gy
Survey of Neural Text Representation Models
2020
Information
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Furthermore, we categorize these models by representation level, input level, model type, and model supervision. ...
Tree-LSTM performs better than an RecNN with basic RNNs because of the same reasons LSTM outperforms RNN. ...
Tree-LSTM [75] introduced a generalization of LSTM to tree-structured network topologies. The model is trained on a supervised sentiment task and it requires parse trees upon input. ...
doi:10.3390/info11110511
fatcat:veamykmme5cm5jhsllyc4xl7ma
Ensemble deep learning: A review
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Other Versions
like bagging, boosting and stacking, negative correlation based deep ensemble models, explicit/implicit ensembles, homogeneous /heterogeneous ensemble, decision fusion strategies, unsupervised, semi-supervised ...
Stochastic depth is an improvement on ResNet [13] wherein residual blocks are randomly dropped during training and bypassing these transformation blocks connections via skip connections. ...
Let h j be the hypothesis generated on the training data D evaluated on test data (x, t), mathematically, h j (t|x) = P[y|x, h j , D]. ...
arXiv:2104.02395v2
fatcat:lq73jqso5vadvnqfnnmw4zul4q
FoldingZero: Protein Folding from Scratch in Hydrophobic-Polar Model
[article]
2018
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
It is trained solely by a reinforcement learning algorithm, which improves HPNet and R-UCT iteratively through iterative policy optimization. ...
In this paper, we propose a novel protein folding framework FoldingZero, self-folding a de novo protein 2D HP structure from scratch based on deep reinforcement learning. ...
In FoldingZero, HPNet is trained in a supervised manner to match the R-UCT search results closely. ...
arXiv:1812.00967v1
fatcat:vjkzsbpjr5fczp4x4y3cod4onu
A Survey of Knowledge Enhanced Pre-trained Models
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Pre-trained models learn contextualized word representations on large-scale text corpus through a self-supervised learning method, which has achieved promising performance after fine-tuning. ...
Pre-trained models with knowledge injection, which we call knowledge enhanced pre-trained models (KEPTMs), possess deep understanding and logical reasoning and introduce interpretability to some extent ...
Transformers via a syntax-aware self-attention mechanism. ...
arXiv:2110.00269v2
fatcat:miinw6thcbasbjnvodp5535w5e
A Deep Learning Approach for a Source Code Detection Model Using Self-Attention
2020
Complexity
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Finally, the representation model encodes the sequence of statement vectors via a bidirectional LSTM network, which is a classical deep learning framework, with a self-attention layer and outputs a vector ...
The representation model firstly transforms the source code into an abstract syntactic tree and splits it into a sequence of statement trees; then, it encodes each of the statement trees with a deep-first ...
It gives a reasonable prediction of code clone by comparing the code similarity in a supervised learning way. ...
doi:10.1155/2020/5027198
doaj:a8882994dfc040c8be3d2f74a2cc9ca6
fatcat:3khymjwlzngwbnvv3h6zj7ym5y
Contrastive Self-supervised Neural Architecture Search
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Note that this fulltext copy is not of the "primary" version of this work. The version it corresponds to is:
2021 pre-print arXiv:2102.10557v2 fatcat:mbsrvqdxp5am3pxo5trxrxqhemOther Versions
First, using only a small amount of unlabeled train data under contrastive self-supervised learning allow us to search on a more extensive search space, discovering better neural architectures without ...
Finally, we tackle the inherent discrete search space of the NAS problem by sequential model-based optimization via the tree-parzen estimator (SMBO-TPE), enabling us to reduce the computational expense ...
Hence, it is not reasonable to compare supervised NAS and self-supervised NAS based on the test accuracy. ...
arXiv:2102.10557v3
fatcat:3qtpuha3njauhjpl72p7j4gyom
Diving Deep into Deep Learning:History, Evolution, Types and Applications
2020
VOLUME-8 ISSUE-10, AUGUST 2019, REGULAR ISSUE
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
While machine learning is busy in supervised and unsupervised methods, deep learning continues its motivation for replicating the human nervous system by incorporating advanced types of Neural Networks ...
Skip-gram is a model for training word embedding algorithms. ...
When it is trained without supervision, it can learn to probabilistically restrict its inputs and acts as feature detectors and with supervision to perform classification [38] . ...
doi:10.35940/ijitee.a4865.019320
fatcat:orn2asvoxfaxvlc5iv7kec4nm4
Self-Supervised Damage-Avoiding Manipulation Strategy Optimization via Mental Simulation
[article]
2017
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
Other Versions
This learned manipulation strategy is continuously refined in a self-supervised optimization cycle dur- ing load-free times of the system. ...
Such a simulation-in-the-loop setup is commonly known as mental simulation and allows for efficient, fully automatic generation of training data as opposed to classical supervised learning approaches. ...
classifier and optimize them in a self-supervised manner. ...
arXiv:1712.07452v1
fatcat:n366fugsmbctxhqm2q6vc6musa
Semi-supervised emotion lexicon expansion with label propagation and specialized word embeddings
[article]
2017
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
In a similar attempt, we learn task-specific word embeddings via a supervised task. ...
As a consequence, the accuracy of a self-training classifier decrements at each iteration. A different method is transductive inference (Vapnik and Vapnik, 1998) . ...
arXiv:1708.03910v1
fatcat:uw3yfzjjfbfxldqmpoybfifzz4
Solos: A Dataset for Audio-Visual Music Analysis
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
, cross-modal correspondences, cross-modal generation and, in general, any audio-visual self-supervised task. ...
URMP was intented to be used for source separation, thus, we evaluate the performance on the URMP dataset of two different source-separation models trained on Solos. ...
Besides, it is possible to carry out self-supervised tasks in which one modality supervises the other one. This entails another research field, the cross-modal correspondence (CMC). ...
arXiv:2006.07931v2
fatcat:3fwx6dxifvczlp445ejr72fohu
Machine Learning on Graphs: A Model and Comprehensive Taxonomy
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Note that this fulltext copy is not of the "primary" version of this work. The version it corresponds to is:
2020 pre-print arXiv:2005.03675v1 fatcat:2nwx74fymzacvka2qq5lunoo3uOther Versions
Specifically, we propose a Graph Encoder Decoder Model (GRAPHEDM), which generalizes popular algorithms for semi-supervised learning on graphs (e.g. ...
The second, graph regularized neural networks, leverages graphs to augment neural network losses with a regularization objective for semi-supervised learning. ...
The attention parameters are trained through backpropagation, and the GAT self-attention mechanism is: g k (H ) = LeakyReLU(H B b 0 ⊕ b 1 BH ) where ⊕ indicates summation of row and column vectors with ...
arXiv:2005.03675v3
fatcat:6eoicgprdvfbze732nsmpaumqe
Synergy of physics-based reasoning and machine learning in biomedical applications: towards unlimited deep learning with limited data
2019
Advances in Physics: X
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Beyond obvious use in initial-factor selection, existing simplified models are effectively employed for generation of realistic synthetic data for later DNN pre-training. ...
We outline our hybrid framework that leverages existing domain-expert models/knowledge, boosting-like model combination, DNN-based DL and other machine learning algorithms for drastic reduction of training-data ...
After that, DNN could be further fine-tuned via supervised training using available labeled data. ...
doi:10.1080/23746149.2019.1582361
fatcat:wkmef4jmgreurnseofsaqa5dva
« Previous
Showing results 1 — 15 out of 3,027 results