786 Hits in 2.5 sec

Deep Multimodal Image-Repurposing Detection [article]

Ekraam Sabir, Wael AbdAlmageed, Yue Wu, Prem Natarajan
2018 pre-print
We present the Multimodal Entity Image Repurposing (MEIR) dataset, a substantially challenging dataset over that which has been previously available to support research into image repurposing detection  ...  We also present a novel, end-to-end, deep multimodal learning model for assessing the integrity of an image by combining information extracted from the image with related information from a knowledge base  ...  We propose a deep multimodal, multi-task learning model for image repurposing detection, as illustrated in Figure 3 .  ... 
doi:10.1145/3240508.3240707 arXiv:1808.06686v1 fatcat:iplva32l6ncr5drg6dxdgbtnqm

MEG: Multi-Evidence GNN for Multimodal Semantic Forensics [article]

Ekraam Sabir, Ayush Jaiswal, Wael AbdAlmageed, Prem Natarajan
2020 arXiv   pre-print
Recent research has centered the problem around images, calling it image repurposing -- where a digitally unmanipulated image is semantically misrepresented by means of its accompanying multimodal metadata  ...  Fake news often involves semantic manipulations across modalities such as image, text, location etc and requires the development of multimodal semantic forensics for its detection.  ...  [6] introduced the multimodal entity image repurposing (MEIR) dataset with challenging manipulations and presented a deep multimodal model (DMM) which achieved state-of-the-art performance on MEIR.  ... 
arXiv:2011.11286v1 fatcat:whw4lz27c5exnnyfvl2rqxv7za

Multimedia Semantic Integrity Assessment Using Joint Embedding Of Images And Text

Ayush Jaiswal, Ekraam Sabir, Wael AbdAlmageed, Premkumar Natarajan
2017 Proceedings of the 2017 ACM on Multimedia Conference - MM '17  
We construct a joint embedding of images and captions with deep multimodal representation learning on the reference dataset in a framework that also provides image-caption consistency scores (ICCSs).  ...  Such multimodal data packages are prone to manipulations, where a subset of these modalities can be altered to misrepresent or repurpose data packages, with possible malicious intent.  ...  Our method combines deep multimodal representation learning with outlier detection methods to assess whether a caption is consistent with the image in its package.  ... 
doi:10.1145/3123266.3123385 dblp:conf/mm/JaiswalSAN17 fatcat:aq2sifpg6ncy3os42i5uvlp43a

Accelerating bioinformatics research with International Conference on Intelligent Biology and Medicine 2020

Yan Guo, Li Shen, Xinghua Shi, Kai Wang, Yulin Dai, Zhongming Zhao
2020 BMC Bioinformatics  
These 12 manuscripts cover a wide range of bioinformatics topics including network analysis, imaging analysis, machine learning, gene expression analysis, and sequence analysis.  ...  Jo et al. published "Deep learning detection of informative features in tau PET for Alzheimer's disease classification" [9] , in which the authors developed a deep learning-based framework to identify  ...  In this special issue, four studies covering imaging and machine learning analysis were included, two of them applied neural network-based deep learning techniques in imaging analysis.  ... 
doi:10.1186/s12859-020-03890-y pmid:33371868 fatcat:oamp777vwfcn5ojliuvb22huu4

A Review of Web Infodemic Analysis and Detection Trends across Multi-modalities using Deep Neural Networks [article]

Chahat Raj, Priyanka Meel
2021 arXiv   pre-print
Fake news detection is one of the most analyzed and prominent areas of research. These detection techniques apply popular machine learning and deep learning algorithms.  ...  This review primarily deals with multi-modal fake news detection techniques that include images, videos, and their combinations with text.  ...  detection, object-removal detection, repurposing detection, and other similar editing detections under image forensics.  ... 
arXiv:2112.00803v1 fatcat:twppg5v37bdozcdloaa6zfk7s4

QuTI! Quantifying Text-Image Consistency in Multimodal Documents [article]

Matthias Springstein and Eric Müller-Budack and Ralph Ewerth
2021 arXiv   pre-print
Typically, multimodal information, e.g., image and text is used to convey information more effectively and to attract attention.  ...  , or other multimodal documents.  ...  This allows us to compare visual features extracted by suitable deep learning approaches of the reference images to the query (news) image.  ... 
arXiv:2104.13748v1 fatcat:2zoemtp5qzeivco2avabnznzeq

NewsCLIPpings: Automatic Generation of Out-of-Context Multimodal Media [article]

Grace Luo, Trevor Darrell, Anna Rohrbach
2021 arXiv   pre-print
While some prior datasets for detecting image-text inconsistency generate samples via text manipulation, we propose a dataset where both image and text are unmanipulated but mismatched.  ...  Our large-scale automatically generated NewsCLIPpings Dataset: (1) demonstrates that machine-driven image repurposing is now a realistic threat, and (2) provides samples that represent challenging instances  ...  Some earlier proposed datasets for detecting multimodal misinformation are MultimodAl Information Manipulation dataset (MAIM) (Jaiswal et al., 2017) and Multimodal Entity Image Repurposing (MEIR) (Sabir  ... 
arXiv:2104.05893v2 fatcat:e4fax5rnafdg5hg2p24rjhtwxy

Repurposing Molecular Imaging and Sensing for Cancer Image-Guided Surgery

Suman Mondal, Christine O'Brien, Kevin Bishop, Ryan Fields, Julie Margenthaler, Samuel Achilefu
2020 Journal of Nuclear Medicine  
Cancer imaging, in particular, has leveraged advances in molecular imaging agents and technology to improve the accuracy of tumor detection, interrogate tumor heterogeneity, monitor treatment response,  ...  Salient features of nuclear, optical, and multimodal approaches will be discussed, including their strengths, limitations and clinical applications.  ...  Multimodal Optical Multimodal optical imaging combines optical modalities for accurate disease detection intraoperatively ( Fig. 5 ; Supplemental Table 6 ).  ... 
doi:10.2967/jnumed.118.220426 pmid:32303598 pmcid:PMC7413229 fatcat:zh2xg6cg7vhojebbed54vrvbka

A Multimodal Learning to Rank model for Web Pages

2020 International Journal of Engineering and Advanced Technology  
VGG-16 model, pre-trained on ImageNet is used as the image feature extractor. The baseline model which is trained only using textual features is compared against the multimodal LTR.  ...  Researches have proven that a multimodality based search would improve the rank list populated. The multiple modalities considered here are the text on a webpage as well as the images on a webpage.  ...  A deep learning architecture to simulate the human judgment process of judging the relevancy is implemented. They designed a detection strategy to extract relevant contexts.  ... 
doi:10.35940/ijeat.f1442.089620 fatcat:2kfk6m47i5hhnfotnn7enheqzy

Multimodal embedding fusion for robust speaker role recognition in video broadcast

Michael Rouvier, Sebastien Delecraz, Benoit Favre, Meriem Bendris, Frederic Bechet
2015 2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU)  
This paper presents a multimodal fusion of audio, text and image embeddings spaces for speaker role recognition in asynchronous data.  ...  Deep Neural Networks (DNN) approaches offer the ability to learn simultaneously feature representations (embeddings) and classification functions.  ...  MULTIMODAL FUSION There are two approaches commonly used for deep multimodal fusion: early and late fusion.  ... 
doi:10.1109/asru.2015.7404820 dblp:conf/asru/RouvierDFBB15 fatcat:2wqd5ewxizbprakccn2adcujqy

Multimodal Object Detection via Probabilistic Ensembling [article]

Yi-Ting Chen, Jinghao Shi, Zelin Ye, Christoph Mertz, Deva Ramanan, Shu Kong
2022 arXiv   pre-print
Object detection with multimodal inputs can improve many safety-critical systems such as autonomous vehicles (AVs).  ...  We validate ProbEn on two benchmarks containing both aligned (KAIST) and unaligned (FLIR) multimodal images, showing that ProbEn outperforms prior work by more than 13% in relative performance!  ...  Multimodal Detection, particularly with RGB-thermal images, has attracted increasing attention.  ... 
arXiv:2104.02904v3 fatcat:hiupy4nmcbh4dmyzzeum4vwzbe

Media forensics on social media platforms: a survey

Cecilia Pasquini, Irene Amerini, Giulia Boato
2021 EURASIP Journal on Information Security  
Lastly, the work in [76] addresses the typical lack of training data for repurposing detection by proposing an Adversarial Image Repurposing Detection (AIRD) method which does not need repurposing examples  ...  A larger and more realistic dataset called MEIR (Multimodal Entity Image Repurposing) 7 is then collected in [72] , where an improved multimodal representation is proposed and a novel architecture is  ... 
doi:10.1186/s13635-021-00117-2 doaj:06be9b5b0da9456b97060870239d1f24 fatcat:fquen2fiyfbazo5b5ydvz3ddae

Multimodal Analytics for Real-world News using Measures of Cross-modal Entity Consistency [article]

Eric Müller-Budack, Jonas Theiner, Sebastian Diering, Maximilian Idahl, Ralph Ewerth
2020 arXiv   pre-print
In this paper, we introduce a novel task of cross-modal consistency verification in real-world news and present a multimodal approach to quantify the entity coherence between image and text.  ...  In some cases such measures could give hints to detect fake news, which is an increasingly important topic in today's society.  ...  In Section 2, we review related work and focus on multimodal image repurposing detection. Our framework to automatically verify cross-modal relations is described in Section 3.  ... 
arXiv:2003.10421v2 fatcat:bqrn4xzc4zhovohco7wwx74xby

Multimodal news analytics using measures of cross-modal entity and context consistency

Eric Müller-Budack, Jonas Theiner, Sebastian Diering, Maximilian Idahl, Sherzod Hakimov, Ralph Ewerth
2021 International Journal of Multimedia Information Retrieval  
In this paper, we present a multimodal approach to quantify the entity coherence between image and text in real-world news.  ...  In some cases such measures might give hints to detect fake news, which is an increasingly important topic in today's society.  ...  The images or other third party material  ... 
doi:10.1007/s13735-021-00207-4 fatcat:bhi7vavllnft3dwx6rw57eqckq

The Power of Computational Intelligence Methods in the Containment of COVID-19 Pandemic from Detection to Recovery [chapter]

Abdullahi Isa, Barka Piyinkir Ndahi
2021 Viral Outbreaks [Working Title]  
This research will focus on the use of computational intelligence methods in understanding the infection, accelerating drugs and treatments research, detecting, diagnosis, and predicting the virus, surveillance  ...  The results for detection accuracy of 92.86% on X-ray images were obtained by the proposed model.  ...  Recently, several pieces of research have been conducted from the digitized image using neural network (CNN) to detect and diagnose COVID-19 [35, 36] .  ... 
doi:10.5772/intechopen.98931 fatcat:bgbimrqkkbht7lvols65vddlsy
« Previous Showing results 1 — 15 out of 786 results