Filters








3,828 Hits in 6.5 sec

Deep Modality Invariant Adversarial Network for Shared Representation Learning

Tatsuya Harada, Kuniaki Saito, Yusuke Mukuta, Yoshitaka Ushiku
2017 2017 IEEE International Conference on Computer Vision Workshops (ICCVW)  
We call our method the Deep Modality Invariant Adversarial Network (DeMIAN). We demonstrate the effectiveness of our method in experiments.  ...  In this work, we propose a novel method to learn the mapping to the common space wherein different modalities have the same information for shared representation learning.  ...  Conclusion In this paper, we proposed a novel method to learn modality-invariant representations for shared representation learning, called the Deep Modal Invariant Adversarial Network (DeMIAN).  ... 
doi:10.1109/iccvw.2017.311 dblp:conf/iccvw/HaradaSMU17 fatcat:ys3e7grypjctxlczcu3j24oode

DeMIAN: Deep Modality Invariant Adversarial Network [article]

Kuniaki Saito, Yusuke Mukuta, Yoshitaka Ushiku, Tatsuya Harada
2016 arXiv   pre-print
In particular, we propose a novel algorithm for modality-invariant representation learning, named Deep Modality Invariant Adversarial Network (DeMIAN), which utilizes the idea of Domain Adaptation (DA)  ...  Using the modality-invariant representations learned by DeMIAN, we achieved better classification accuracy than with the state-of-the-art methods, especially for some benchmark datasets of zero-shot learning  ...  This work was funded by ImPACT Program of Council for Science, Technology and Innovation (Cabinet Office, Government of Japan) and supported by CREST, JST.  ... 
arXiv:1612.07976v2 fatcat:dyr3t25yfvf3vm2px2asfylv64

Survey on Deep Multi-modal Data Analytics: Collaboration, Rivalry and Fusion [article]

Yang Wang
2020 arXiv   pre-print
Recently, deep neural networks have exhibited as a powerful architecture to well capture the nonlinear distribution of high-dimensional multimedia data, so naturally does for multi-modal data.  ...  Throughout this survey, we further indicate that the critical components for this field go to collaboration, adversarial competition and fusion over multi-modal spaces.  ...  Deep Modality Invariant Adversarial Network (DeMIAN) [38] aimed to learn modality-invariant representations from paired modalities, while enabling to train a classifier on source modality samples, which  ... 
arXiv:2006.08159v1 fatcat:g4467zmutndglmy35n3eyfwxku

Deep multimodal representation learning: a survey

Wenzhong Guo, Jianwen Wang, Shiping Wanga
2019 IEEE Access  
INDEX TERMS Multimodal representation learning, multimodal deep learning, deep multimodal fusion, multimodal translation, multimodal adversarial learning.  ...  This paper highlights on the key issues of newly developed technologies, such as encoder-decoder model, generative adversarial networks, and attention mechanism in a multimodal representation learning  ...  Two methods used for improving modality-invariant property via adversarial learning.  ... 
doi:10.1109/access.2019.2916887 fatcat:ms4wcgl5rncsbiywz27uss4ysq

Unsupervised Learning of View-invariant Action Representations [article]

Junnan Li, Yongkang Wong, Qi Zhao, Mohan S. Kankanhalli
2018 arXiv   pre-print
In addition, we propose a view-adversarial training method to enhance learning of view-invariant features.  ...  By learning to extrapolate cross-view motions, the representation can capture view-invariant motion dynamics which is discriminative for the action.  ...  The second category uses deep networks to jointly learn feature representation and classifier.  ... 
arXiv:1809.01844v1 fatcat:u4w3kzv6ofaplkawtapmkscisi

Adversarial Discriminative Heterogeneous Face Recognition [article]

Lingxiao Song and Man Zhang and Xiang Wu and Ran He
2017 arXiv   pre-print
These two losses enhance domain-invariant feature learning and modality independent noise removing.  ...  This framework integrates cross-spectral face hallucination and discriminative feature learning into an end-to-end adversarial network.  ...  In , Restricted Boltzmann Machines (RBMs) is used to learn a shared representation between different modalities.  ... 
arXiv:1709.03675v1 fatcat:gnjwnkszrfgnlikmla6vgqnqhm

Adversarial Discriminative Domain Adaptation [article]

Eric Tzeng, Judy Hoffman, Kate Saenko, Trevor Darrell
2017 arXiv   pre-print
Adversarial learning methods are a promising approach to training robust deep networks, and can generate complex samples across diverse domains.  ...  We first outline a novel generalized framework for adversarial adaptation, which subsumes recent state-of-the-art approaches as special cases, and we use this generalized view to better relate the prior  ...  Conclusion We have proposed a unified framework for unsupervised domain adaptation techniques based on adversarial learning objectives.  ... 
arXiv:1702.05464v1 fatcat:zzepyxgtyzbanhxrrehknzeexa

Adversarial Discriminative Heterogeneous Face Recognition

Lingxiao Song, Man Zhang, Xiang Wu, Ran He
2018 PROCEEDINGS OF THE THIRTIETH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE AND THE TWENTY-EIGHTH INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE  
These two losses enhance domain-invariant feature learning and modality independent noise removing.  ...  This framework integrates cross-spectral face hallucination and discriminative feature learning into an end-to-end adversarial network.  ...  In (Yi, Lei, and Li 2015) , Restricted Boltzmann Machines (RBMs) is used to learn a shared representation between different modalities.  ... 
doi:10.1609/aaai.v32i1.12291 fatcat:l26znnxzfbds7lpqqlcvmsqkqq

Adversarial Cross-Modal Retrieval

Bokun Wang, Yang Yang, Xing Xu, Alan Hanjalic, Heng Tao Shen
2017 Proceedings of the 2017 ACM on Multimedia Conference - MM '17  
In this paper, we present a novel Adversarial Cross-Modal Retrieval (ACMR) method, which seeks an effective common subspace based on adversarial learning.  ...  The first process, a feature projector, tries to generate a modality-invariant representation in the common subspace and to confuse the other process, modality classifier, which tries to discriminate between  ...  With ACMR, we combine for the first time the concepts of supervised representation learning for cross-modal retrieval and adversarial learning.  ... 
doi:10.1145/3123266.3123326 dblp:conf/mm/WangYXHS17 fatcat:7pbuzhkqsbbt7bmhyj45qazyea

Integrating Information Theory and Adversarial Learning for Cross-modal Retrieval

Wei Chen, Yu Liu, Erwin M. Bakker, Michael S. Lew
2021 Pattern Recognition  
To address these challenges posited by the heterogeneity gap and the semantic gap, we propose integrating Shannon information theory and adversarial learning.  ...  For this purpose, a modality classifier (as a discriminator) is built to distinguish the text and image modalities according to their different statistical properties.  ...  We would like to thank Theodoros Georgiou and Nan Pu for several discussions and thank NVIDIA for the donation of GPU cards.  ... 
doi:10.1016/j.patcog.2021.107983 fatcat:jzx5h5yfzvhvphpzdycowd5c24

Adversarial Discriminative Domain Adaptation

Eric Tzeng, Judy Hoffman, Kate Saenko, Trevor Darrell
2017 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)  
Adversarial learning methods are a promising approach to training robust deep networks, and can generate complex samples across diverse domains.  ...  We then propose a previously unexplored instance of our general framework which combines discriminative modeling, untied weight sharing, and a GAN loss, which we call Adversarial Discriminative Domain  ...  Introduction Deep convolutional networks, when trained on large-scale datasets, can learn representations which are generically usefull across a variety of tasks and visual domains [1, 2] .  ... 
doi:10.1109/cvpr.2017.316 dblp:conf/cvpr/TzengHSD17 fatcat:lpnyganiercatbsofc5z326vhe

Integration of Unpaired Single-cell Chromatin Accessibility and Gene Expression Data via Adversarial Learning [article]

Yang Xu, Andrew Jeremiah Strick
2021 arXiv   pre-print
Deep learning has empowered analysis for single-cell sequencing data in many ways and has generated deep understanding about a range of complex cellular systems.  ...  of high dimensional data that come from different sources and represent cellular systems with different features, there is an equivalent rise and challenge of integrating single-cell sequence across modalities  ...  Unsupervised representation learning Deep metric learning has shown effective representation learning without supervision.  ... 
arXiv:2104.12320v1 fatcat:sb6cokdqpbhm5cyu3nxcgo5f6a

Advancing Medical Imaging Informatics by Deep Learning-Based Domain Adaptation

Anirudh Choudhary, Li Tong, Yuanda Zhu, May D. Wang
2020 IMIA Yearbook of Medical Informatics  
There has been a rapid development of deep learning (DL) models for medical imaging. However, DL requires a large labeled dataset for training the models.  ...  Using adversarial techniques, unsupervised DA has achieved good performance, especially for segmentation tasks.  ...  to learn a domain-invariant feature representation.  ... 
doi:10.1055/s-0040-1702009 pmid:32823306 fatcat:gtlhoh6m3fh4hcumfzdlpdohr4

MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysis [article]

Devamanyu Hazarika, Roger Zimmermann, Soujanya Poria
2020 arXiv   pre-print
The first subspace is modality-invariant, where the representations across modalities learn their commonalities and reduce the modality gap.  ...  In this paper, we aim to learn effective modality representations to aid the process of fusion. We propose a novel framework, MISA, which projects each modality to two distinct subspaces.  ...  Similar to the third category, we also learn common modality-invariant subspaces. However, we do not use adversarial discriminators to learn shared mappings.  ... 
arXiv:2005.03545v3 fatcat:nyomobnpojcefpyllea3scjdpq

Review of Recent Deep Learning Based Methods for Image-Text Retrieval

Jianan Chen, Lu Zhang, Cong Bai, Kidiyo Kpalma
2020 2020 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR)  
In this paper, we highlight key points of recent cross-modal retrieval approaches based on deep-learning, especially in the image-text retrieval context, and classify them into four categories according  ...  Cross-modal retrieval aims to retrieve relevant information across different modalities.  ...  After that, Sarafianos et al. propose Text-Image Modality Adversarial Matching (TIMAM [34] ), which adopts an Adversarial Representation Learning (ARL) framework to learn modality-invariant representations  ... 
doi:10.1109/mipr49039.2020.00042 dblp:conf/mipr/ChenZBK20 fatcat:fps5wiw4ezf7teko3vegaxq4tq
« Previous Showing results 1 — 15 out of 3,828 results