Filters








206,749 Hits in 7.2 sec

Learning Approximate Inference Networks for Structured Prediction [article]

Lifu Tu, Kevin Gimpel
2018 arXiv   pre-print
We replace this use of gradient descent with a neural network trained to approximate structured argmax inference.  ...  Finally, we show how inference networks can replace dynamic programming for test-time inference in conditional random fields, suggestive for their general use for fast inference in structured settings.  ...  We also thank NVIDIA Corporation for donating GPUs used in this research.  ... 
arXiv:1803.03376v1 fatcat:enx3voi2u5anbonxcns5wygqcq

Adversarial Localized Energy Network for Structured Prediction

Pingbo Pan, Ping Liu, Yan Yan, Tianbao Yang, Yi Yang
2020 PROCEEDINGS OF THE THIRTIETH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE AND THE TWENTY-EIGHTH INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE  
On the one hand, our modified inference network can boost the efficiency by predicting good initializations and reducing the searching space for the inference process; On the other hand, inheriting the  ...  This paper focuses on energy model based structured output prediction.  ...  One reason is that the predicted output structure of the inference network is already close to the optimal structure.  ... 
doi:10.1609/aaai.v34i04.5982 fatcat:7ldu6jwydfa3nnxmshykmgsapa

End-to-end learning potentials for structured attribute prediction [article]

Kota Yamaguchi, Takayuki Okatani, Takayuki Umeda, Kazuhiko Murasaki, Kyoko Sudo
2017 arXiv   pre-print
We present a structured inference approach in deep neural networks for multiple attribute prediction.  ...  We model potential functions by deep neural networks and apply the sum-product algorithm to solve for the approximate marginal distribution in feed-forward networks.  ...  ) marginal inference on a sparse factor graph in a feed-forward network, and also to enable back propagation for end-toend learning. • We empirically show that the structured inference can improve attribute  ... 
arXiv:1708.01892v1 fatcat:km5n6eay6ra5bjju5yqnzucfsu

Challenges and Opportunities in Approximate Bayesian Deep Learning for Intelligent IoT Systems [article]

Meet P. Vadera, Benjamin M. Marlin
2021 arXiv   pre-print
In this paper, we present a range of approximate Bayesian inference methods for supervised deep learning and highlight the challenges and opportunities when applying these methods on current edge hardware  ...  Approximate Bayesian deep learning methods hold significant promise for addressing several issues that occur when deploying deep learning components in intelligent systems, including mitigating the occurrence  ...  BDK approximates the posterior predictive distribution by learning an auxiliary neural network model to compress the Monte Carlo approximation to the posterior predictive distribution E pMC (θ|D,λ) [p(  ... 
arXiv:2112.01675v1 fatcat:okknsw5gifhl7ghzt4cnuchuje

Structured Prediction Energy Networks [article]

David Belanger, Andrew McCallum
2016 arXiv   pre-print
We introduce structured prediction energy networks (SPENs), a flexible framework for structured prediction.  ...  Overall, deep learning provides remarkable tools for learning features of the inputs to a prediction problem, and this work extends these techniques to learning features of structured outputs.  ...  Thanks to Luke Vilnis for helpful advice.  ... 
arXiv:1511.06350v3 fatcat:bql7nutfbjeehfhw6ynddi37si

Benchmarking Approximate Inference Methods for Neural Structured Prediction [article]

Lifu Tu, Kevin Gimpel
2019 arXiv   pre-print
Exact structured inference with neural network scoring functions is computationally challenging but several methods have been proposed for approximating inference.  ...  We find further benefit by combining inference networks and gradient descent, using the former to provide a warm start for the latter.  ...  Acknowledgments We would like to thank Ke Li for suggesting experiments that combine inference networks and gradient descent, the anonymous reviewers for their feedback, and NVIDIA for donating GPUs used  ... 
arXiv:1904.01138v2 fatcat:jqzp6lmf55hfjb3wzud4vm7ani

Learning Discriminators as Energy Networks in Adversarial Learning [article]

Pingbo Pan, Yan Yan, Tianbao Yang, Yi Yang
2018 arXiv   pre-print
We propose a novel framework for structured prediction via adversarial learning.  ...  Existing adversarial learning methods involve two separate networks, i.e., the structured prediction models and the discriminative models, in the training.  ...  There is a rising interest in energy-based structured prediction (Zheng et al., 2015; Chen et al., 2015; Song et al., 2016) .  ... 
arXiv:1810.01152v1 fatcat:4conkrbr2rbdjo4ozohbemu2om

Deep Component Analysis via Alternating Direction Neural Networks [article]

Calvin Murdock, Ming-Fang Chang, Simon Lucey
2018 arXiv   pre-print
By interpreting feed-forward networks as single-iteration approximations of inference in our model, we provide both a novel theoretical perspective for understanding them and a practical technique for  ...  For inference, we propose a differentiable optimization algorithm implemented using recurrent Alternating Direction Neural Networks (ADNNs) that enable parameter learning using standard backpropagation  ...  Learning by Backpropagation With DeepCA inference approximated by differentiable ADNNs, the model parameters can be learned in the same way as standard feed-forward networks.  ... 
arXiv:1803.06407v1 fatcat:tfivbuxbvbfc5lepgeglb5gpru

Non-parametric Structured Output Networks

Andreas M. Lehrmann, Leonid Sigal
2017 Neural Information Processing Systems  
End-to-end training methods for models with structured graphical dependencies on top of neural predictions have recently emerged as a principled way of combining these two paradigms.  ...  Deep neural networks (DNNs) and probabilistic graphical models (PGMs) are the two main tools for statistical modeling.  ...  Our experiments showed that non-parametric structured output networks are necessary for both effective learning of multimodal posteriors and efficient inference of complex statistics in them.  ... 
dblp:conf/nips/LehrmannS17 fatcat:rdf6i2i3k5gadjlpq3zvpwav6q

Predict and Constrain: Modeling Cardinality in Deep Structured Prediction [article]

Nataly Brukhim, Amir Globerson
2018 arXiv   pre-print
Many machine learning problems require the prediction of multi-dimensional labels. Such structured prediction models can benefit from modeling dependencies between labels.  ...  Recently, several deep learning approaches to structured prediction have been proposed. Here we focus on capturing cardinality constraints in such models.  ...  Hence, we break our inference process into two complementary components: we first estimate the label set cardinality for a given input using a learned neural network, and then predict a label that satisfies  ... 
arXiv:1802.04721v1 fatcat:ifnrvz3kczg5xm24ztfxd75k2u

Learning Energy-Based Approximate Inference Networks for Structured Applications in NLP [article]

Lifu Tu
2021 arXiv   pre-print
Structured prediction in natural language processing (NLP) has a long history. The complex models of structured application come at the difficulty of learning and inference.  ...  We provide a learning framework for complicated structured models as well as an inference method with a better speed/accuracy/search error trade-off.  ...  Inference Networks This chapter describes our contributions to approximate structure inference for structured tasks.  ... 
arXiv:2108.12522v1 fatcat:4ybm5dh5zza5znsfqyyddji6za

GENN: Predicting Correlated Drug-drug Interactions with Graph Energy Neural Networks [article]

Tengfei Ma, Junyuan Shang, Cao Xiao, Jimeng Sun
2019 arXiv   pre-print
We formulate the DDI prediction task as a structure prediction problem and introduce a new energy-based model where the energy function is defined by graph neural networks.  ...  Recently graph neural networks have achieved great success in this task by modeling drugs as nodes and drug-drug interactions as links and casting DDI predictions as link prediction problems.  ...  We designed one cost-augmented inference network to approximate the output in training and one test inference network to approximate the output in testing.  ... 
arXiv:1910.02107v2 fatcat:jwgbkbvlvvgh7f7pjpqn4k4tp4

Modeling Signal Transduction from Protein Phosphorylation to Gene Expression

Chunhui Cai, Lujia Chen, Xia Jiang, Xinghua Lu
2014 Cancer Informatics  
We designed an efficient inference algorithm that incorporated the prior knowledge of pathways and searched for a network structure in a data-driven manner. results: We applied our method to infer rat  ...  bAckground: Signaling networks are of great importance for us to understand the cell's regulatory mechanism.  ...  Gregory Cooper and Sognjian Lu for their constructive discussions during the project. The authors also would like to thank Mr. Kevin Lu for proofreading the final manuscript.  ... 
doi:10.4137/cin.s13883 pmid:25392684 pmcid:PMC4216050 fatcat:7qeilblut5fv7aphwqducghshq

Structured Output Learning with Conditional Generative Flows

You Lu, Bert Huang
2020 PROCEEDINGS OF THE THIRTIETH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE AND THE TWENTY-EIGHTH INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE  
In this paper, we propose conditional Glow (c-Glow), a conditional generative flow for structured output learning.  ...  For many models, computing the likelihood is intractable. These models are therefore hard to train, requiring the use of surrogate objectives or variational inference to approximate likelihood.  ...  Acknowledgments We thank NVIDIA's GPU Grant Program and Amazon's AWS Cloud Credits for Research program for their support.  ... 
doi:10.1609/aaai.v34i04.5940 fatcat:jk67kkunibfbrlcxjun36mx4yy

Input Convex Neural Networks [article]

Brandon Amos, Lei Xu, J. Zico Kolter
2017 arXiv   pre-print
The networks allow for efficient inference via optimization over some inputs to the network given others, and can be applied to settings including structured prediction, data imputation, reinforcement  ...  In this paper we lay the basic groundwork for these models, proposing methods for inference, optimization and learning, and analyze their representational power.  ...  We also thank David Belanger for helpful discussions.  ... 
arXiv:1609.07152v3 fatcat:fke2eonbyjbcbiqfvudec4ov3q
« Previous Showing results 1 — 15 out of 206,749 results