30 Hits in 1.2 sec

Tensor-Train Recurrent Neural Networks for Video Classification [article]

Yinchong Yang, Denis Krompass, Volker Tresp
2017 arXiv   pre-print
The Recurrent Neural Networks and their variants have shown promising performances in sequence modeling tasks such as Natural Language Processing. These models, however, turn out to be impractical and difficult to train when exposed to very high-dimensional inputs due to the large input-to-hidden weight matrix. This may have prevented RNNs' large-scale application in tasks that involve very high input dimensions such as video modeling; current approaches reduce the input dimensions using
more » ... feature extractors. To address this challenge, we propose a new, more general and efficient approach by factorizing the input-to-hidden weight matrix using Tensor-Train decomposition which is trained simultaneously with the weights themselves. We test our model on classification tasks using multiple real-world video datasets and achieve competitive performances with state-of-the-art models, even though our model architecture is orders of magnitude less complex. We believe that the proposed approach provides a novel and fundamental building block for modeling high-dimensional sequential data with RNN architectures and opens up many possibilities to transfer the expressive and advanced architectures from other domains such as NLP to modeling high-dimensional sequential data.
arXiv:1707.01786v1 fatcat:tppd5eyfeffqdffjkehqfe5rai

ARCADe: A Rapid Continual Anomaly Detector [article]

Ahmed Frikha, Denis Krompaß, Volker Tresp
2020 arXiv   pre-print
Although continual learning and anomaly detection have separately been well-studied in previous works, their intersection remains rather unexplored. The present work addresses a learning scenario where a model has to incrementally learn a sequence of anomaly detection tasks, i.e. tasks from which only examples from the normal (majority) class are available for training. We define this novel learning problem of continual anomaly detection (CAD) and formulate it as a meta-learning problem.
more » ... r, we propose A Rapid Continual Anomaly Detector (ARCADe), an approach to train neural networks to be robust against the major challenges of this new learning problem, namely catastrophic forgetting and overfitting to the majority class. The results of our experiments on three datasets show that, in the CAD problem setting, ARCADe substantially outperforms baselines from the continual learning and anomaly detection literature. Finally, we provide deeper insights into the learning strategy yielded by the proposed meta-learning algorithm.
arXiv:2008.04042v2 fatcat:ti5swtcwxzdcpo4y7j3ftlv74a

Querying Factorized Probabilistic Triple Databases [chapter]

Denis Krompaß, Maximilian Nickel, Volker Tresp
2014 Lecture Notes in Computer Science  
An increasing amount of data is becoming available in the form of large triple stores, with the Semantic Web's linked open data cloud (LOD) as one of the most prominent examples. Data quality and completeness are key issues in many community-generated data stores, like LOD, which motivates probabilistic and statistical approaches to data representation, reasoning and querying. In this paper we address the issue from the perspective of probabilistic databases, which account for uncertainty in
more » ... data via a probability distribution over all database instances. We obtain a highly compressed representation using the recently developed RESCAL approach and demonstrate experimentally that efficient querying can be obtained by exploiting inherent features of RESCAL via sub-query approximations of deterministic views.
doi:10.1007/978-3-319-11915-1_8 fatcat:j6cakwd6avgu7ln4nt7rmnnyea

Towards Data-Free Domain Generalization [article]

Ahmed Frikha, Haokun Chen, Denis Krompaß, Thomas Runkler, Volker Tresp
2021 arXiv   pre-print
In this work, we investigate the unexplored intersection of domain generalization and data-free learning. In particular, we address the question: How can knowledge contained in models trained on different source data domains can be merged into a single model that generalizes well to unseen target domains, in the absence of source and target domain data? Machine learning models that can cope with domain shift are essential for for real-world scenarios with often changing data distributions.
more » ... domain generalization methods typically rely on using source domain data, making them unsuitable for private decentralized data. We define the novel problem of Data-Free Domain Generalization (DFDG), a practical setting where models trained on the source domains separately are available instead of the original datasets, and investigate how to effectively solve the domain generalization problem in that case. We propose DEKAN, an approach that extracts and fuses domain-specific knowledge from the available teacher models into a student model robust to domain shift. Our empirical evaluation demonstrates the effectiveness of our method which achieves first state-of-the-art results in DFDG by significantly outperforming ensemble and data-free knowledge distillation baselines.
arXiv:2110.04545v2 fatcat:yyvsy5rpordbpk5rugeb5gll34

Type-Constrained Representation Learning in Knowledge Graphs [chapter]

Denis Krompaß, Stephan Baier, Volker Tresp
2015 Lecture Notes in Computer Science  
efficiency of algorithm • A local closed-world assumption is a powerful but simple alternative in case type-constraints are absent or fuzzy • Both approaches can be combined  ... 
doi:10.1007/978-3-319-25007-6_37 fatcat:c6wqpgoeffbtvosxv3vajj2shq

Type-Constrained Representation Learning in Knowledge Graphs [article]

Denis Krompaß and Stephan Baier and Volker Tresp
2015 arXiv   pre-print
Code and datasets will be available from∼krompass/ 4, canonicalized datasets: mapping-basedproperties  ... 
arXiv:1508.02593v2 fatcat:wn2vujqvpjdkrosxaqevl7bepe

FRAug: Tackling Federated Learning with Non-IID Features via Representation Augmentation [article]

Haokun Chen, Ahmed Frikha, Denis Krompass, Volker Tresp
2022 arXiv   pre-print
Federated Learning (FL) is a decentralized learning paradigm in which multiple clients collaboratively train deep learning models without centralizing their local data and hence preserve data privacy. Real-world applications usually involve a distribution shift across the datasets of the different clients, which hurts the generalization ability of the clients to unseen samples from their respective data distributions. In this work, we address the recently proposed feature shift problem where
more » ... clients have different feature distributions while the label distribution is the same. We propose Federated Representation Augmentation (FRAug) to tackle this practical and challenging problem. Our approach generates synthetic client-specific samples in the embedding space to augment the usually small client datasets. For that, we train a shared generative model to fuse the clients' knowledge, learned from different feature distributions, to synthesize client-agnostic embeddings, which are then locally transformed into client-specific embeddings by Representation Transformation Networks (RTNets). By transferring knowledge across the clients, the generated embeddings act as a regularizer for the client models and reduce overfitting to the local original datasets, hence improving generalization. Our empirical evaluation on multiple benchmark datasets demonstrates the effectiveness of the proposed method, which substantially outperforms the current state-of-the-art FL methods for non-IID features, including PartialFed and FedBN.
arXiv:2205.14900v1 fatcat:h7kpgis4crd5pao3g7hqbiuhji

Few-Shot One-Class Classification via Meta-Learning [article]

Ahmed Frikha, Denis Krompaß, Hans-Georg Köpken, Volker Tresp
2021 arXiv   pre-print
Although few-shot learning and one-class classification (OCC), i.e., learning a binary classifier with data from only one class, have been separately well studied, their intersection remains rather unexplored. Our work addresses the few-shot OCC problem and presents a method to modify the episodic data sampling strategy of the model-agnostic meta-learning (MAML) algorithm to learn a model initialization particularly suited for learning few-shot OCC tasks. This is done by explicitly optimizing
more » ... r an initialization which only requires few gradient steps with one-class minibatches to yield a performance increase on class-balanced test data. We provide a theoretical analysis that explains why our approach works in the few-shot OCC scenario, while other meta-learning algorithms fail, including the unmodified MAML. Our experiments on eight datasets from the image and time-series domains show that our method leads to better results than classical OCC and few-shot classification approaches, and demonstrate the ability to learn unseen tasks from only few normal class samples. Moreover, we successfully train anomaly detectors for a real-world application on sensor readings recorded during industrial manufacturing of workpieces with a CNC milling machine, by using few normal examples. Finally, we empirically demonstrate that the proposed data sampling technique increases the performance of more recent meta-learning algorithms in few-shot OCC and yields state-of-the-art results in this problem setting.
arXiv:2007.04146v2 fatcat:743t776ymva7fnacif7wx7c6qm

Predicting the Co-Evolution of Event and Knowledge Graphs [article]

Cristóbal Esteban and Volker Tresp and Yinchong Yang and Stephan Baier and Denis Krompaß
2015 arXiv   pre-print
Embedding learning, a.k.a. representation learning, has been shown to be able to model large-scale semantic knowledge graphs. A key concept is a mapping of the knowledge graph to a tensor representation whose entries are predicted by models using latent representations of generalized entities. Knowledge graphs are typically treated as static: A knowledge graph grows more links when more facts become available but the ground truth values associated with links is considered time invariant. In
more » ... paper we address the issue of knowledge graphs where triple states depend on time. We assume that changes in the knowledge graph always arrive in form of events, in the sense that the events are the gateway to the knowledge graph. We train an event prediction model which uses both knowledge graph background information and information on recent events. By predicting future events, we also predict likely changes in the knowledge graph and thus obtain a model for the evolution of the knowledge graph as well. Our experiments demonstrate that our approach performs well in a clinical application, a recommendation engine and a sensor network application.
arXiv:1512.06900v1 fatcat:zcvnp27erfatjaf4eanvrjozpq

Pruning Rogue Taxa Improves Phylogenetic Accuracy: An Efficient Algorithm and Webservice

Andre J. Aberer, Denis Krompass, Alexandros Stamatakis
2012 Systematic Biology  
The presence of rogue taxa (rogues) in a set of trees can frequently have a negative impact on the results of a bootstrap analysis (e.g., the overall support in consensus trees). We introduce an efficient graph-based algorithm for rogue taxon identification as well as an interactive webservice implementing this algorithm. Compared with our previous method, the new algorithm is up to 4 orders of magnitude faster, while returning qualitatively identical results. Because of this significant
more » ... ment in scalability, the new algorithm can now identify substantially more complex and computeintensive rogue taxon constellations. On a large and diverse collection of real-world data sets, we show that our method yields better supported reduced/pruned consensus trees than any competing rogue taxon identification method. Using the parallel version of our open-source code, we successfully identified rogue taxa in a set of 100 trees with 116 334 taxa each. For simulated data sets, we show that when removing/pruning rogue taxa with our method from a tree set, we consistently obtain bootstrap consensus trees as well as maximum-likelihood trees that are topologically closer to the respective true trees. [Bootstrap support; consensus tree; phylogenetic postanalysis; rogue taxa; software; webservice.]
doi:10.1093/sysbio/sys078 pmid:22962004 pmcid:PMC3526802 fatcat:5posisvad5h4xgtt76jnfv5hsq

Exploiting Latent Embeddings of Nominal Clinical Data for Predicting Hospital Readmission

Denis Krompaß, Cristóbal Esteban, Volker Tresp, Martin Sedlmayr, Thomas Ganslandt
2014 Künstliche Intelligenz  
Hospital readmissions of patients put a high burden not only on the health care system, but also on the patients since complications after discharge generally lead to additional burdens. Estimating the risk of readmission after discharge from inpatient care has been the subject of several publications in recent years. In those publications the authors mostly tried to directly infer the readmission risk (within a certain time frame) from the clinical data recorded in the medical routine such as
more » ... rimary diagnosis, co-morbidities, length of stay, or questionnaires. Instead of using these data directly as inputs for a prediction model, we are exploiting latent embeddings for the nominal parts of the data (e.g. diagnosis and procedure codes). These latent embeddings have been used with great success in the natural language processing domain and can be constructed in a preprocessing step. We show in our experiments, that a prediction model that exploits these latent embeddings can lead to improved readmission predictive models.
doi:10.1007/s13218-014-0344-x fatcat:yhihgdk3hvcnzpgw5j7aobusuu

Large-scale factorization of type-constrained multi-relational data

Denis Krompass, Maximilian Nickel, Volker Tresp
2014 2014 International Conference on Data Science and Advanced Analytics (DSAA)  
The statistical modeling of large multi-relational datasets has increasingly gained attention in recent years. Typical applications involve large knowledge bases like DBpedia, Freebase, YAGO and the recently introduced Google Knowledge Graph that contain millions of entities, hundreds and thousands of relations, and billions of relational tuples. Collective factorization methods have been shown to scale up to these large multirelational datasets, in particular in form of tensor approaches that
more » ... an exploit the highly scalable alternating least squares (ALS) algorithms for calculating the factors. In this paper we extend the recently proposed state-of-the-art RESCAL tensor factorization to consider relational type-constraints. Relational type-constraints explicitly define the logic of relations by excluding entities from the subject or object role. In addition we will show that in absence of prior knowledge about type-constraints, local closedworld assumptions can be approximated for each relation by ignoring unobserved subject or object entities in a relation. In our experiments on representative large datasets (Cora, DBpedia), that contain up to millions of entities and hundreds of typeconstrained relations, we show that the proposed approach is scalable. It further significantly outperforms RESCAL without type-constraints in both, runtime and prediction quality.
doi:10.1109/dsaa.2014.7058046 dblp:conf/dsaa/KrompassNT14 fatcat:vq3fhf4dsnbohheksmymlq4f6a

Predicting Sequences of Clinical Events by Using a Personalized Temporal Latent Embedding Model

Cristobal Esteban, Danilo Schmidt, Denis Krompass, Volker Tresp
2015 2015 International Conference on Healthcare Informatics  
As a result of the recent trend towards digitization -which increasingly affects evidence-based medicine, accountable care, personalized medicine, and medical "Big Data" analysis-growing amounts of clinical data are becoming available for analysis. In this paper, we follow the idea that one can model clinical processes based on clinical data, which can then be the basis for many useful applications. We model the whole clinical evolution of each individual patient, which is composed of thousands
more » ... of events such as ordered tests, lab results and diagnoses. Specifically, we base our work on a dataset provided by the Charité University Hospital of Berlin which is composed of patients that suffered from kidney failure and either obtained an organ transplant or are still waiting for one. These patients face a lifelong treatment and periodic visits to the clinic. Our goal is to develop a system to predict the sequence of events recorded in the electronic medical record of each patient, and thus to develop the basis for a future clinical decision support system. For modelling, we use machine learning approaches which are based on a combination of the embedding of entities and events in a multidimensional latent space, in combination with Neural Network predictive models. Similar approaches have been highly successful in statistical models for recommendation systems, language models, and knowledge graphs. We extend existing embedding models to the clinical domain, in particular with respect to temporal sequences, long-term memories and personalization. We compare the performance of our proposed models with standard approaches such as K-nearest neighbors method, Naive Bayes classifier and Logistic Regression, and obtained favorable results with our proposed model.
doi:10.1109/ichi.2015.23 dblp:conf/ichi/EstebanSKT15 fatcat:wnybkb6lhzetdiqixxwwscgfqy

Performance, Accuracy, and Web Server for Evolutionary Placement of Short Sequence Reads under Maximum Likelihood

Simon A. Berger, Denis Krompass, Alexandros Stamatakis
2011 Systematic Biology  
We present an evolutionary placement algorithm (EPA) and a Web server for the rapid assignment of sequence fragments (short reads) to edges of a given phylogenetic tree under the maximum-likelihood model. The accuracy of the algorithm is evaluated on several real-world data sets and compared with placement by pair-wise sequence comparison, using edit distances and BLAST. We introduce a slow and accurate as well as a fast and less accurate placement algorithm. For the slow algorithm, we develop
more » ... dditional heuristic techniques that yield almost the same run times as the fast version with only a small loss of accuracy. When those additional heuristics are employed, the run time of the more accurate algorithm is comparable with that of a simple BLAST search for data sets with a high number of short query sequences. Moreover, the accuracy of the EPA is significantly higher, in particular when the sample of taxa in the reference topology is sparse or inadequate. Our algorithm, which has been integrated into RAxML, therefore provides an equally fast but more accurate alternative to BLAST for tree-based inference of the evolutionary origin and composition of short sequence reads. We are also actively developing a Web server that offers a freely available service for computing read placements on trees using the EPA. [Maximum likelihood; metagenomics; phylogenetic placement; RAxML; short sequence reads.]
doi:10.1093/sysbio/syr010 pmid:21436105 pmcid:PMC3078422 fatcat:lroxs4lgyrahxjgaj5fwa53iu4

Discovery of New Multi-Level Features for Domain Generalization via Knowledge Corruption [article]

Ahmed Frikha, Denis Krompaß, Volker Tresp
2022 arXiv   pre-print
Machine learning models that can generalize to unseen domains are essential when applied in real-world scenarios involving strong domain shifts. We address the challenging domain generalization (DG) problem, where a model trained on a set of source domains is expected to generalize well in unseen domains without any exposure to their data. The main challenge of DG is that the features learned from the source domains are not necessarily present in the unseen target domains, leading to
more » ... deterioration. We assume that learning a richer set of features is crucial to improve the transfer to a wider set of unknown domains. For this reason, we propose COLUMBUS, a method that enforces new feature discovery via a targeted corruption of the most relevant input and multi-level representations of the data. We conduct an extensive empirical evaluation to demonstrate the effectiveness of the proposed approach which achieves new state-of-the-art results by outperforming 18 DG algorithms on multiple DG benchmark datasets in the DomainBed framework.
arXiv:2109.04320v2 fatcat:ynt6owfaffa5hjrjmesku4hin4
« Previous Showing results 1 — 15 out of 30 results