24,706 Hits in 5.1 sec

A learning optimizer for a federated database management system

Stephan Ewen, Michael Ortega-Binderberger, Volker Markl
2005 Informatik - Forschung und Entwicklung  
We present an approach that extends DB2s learning optimizer to automatically find flaws in statistics on remote data by extending its query feedback loop towards the federated architecture.  ...  Optimizers in modern DBMSs utilize a cost model to choose an efficient query execution plan (QEP) among all possible ones for a given query.  ...  The major difference to the non federated learning optimizer is the query monitoring component.  ... 
doi:10.1007/s00450-005-0206-8 fatcat:xbq2w57pe5fq5m7gpavmdpuopu

Estimating Query Result Sizes for Proxy Caching in Scientific Database Federations

Tanu Malik, Randal Burns, Nitesh Chawla, Alex Szalay
2006 ACM/IEEE SC 2006 Conference (SC'06)  
In a proxy cache for federations of scientific databases it is important to estimate the size of a query before making a caching decision.  ...  CAROT estimates query result sizes by learning the distribution of query results, not by examining or sampling data, but from observing workload.  ...  Such queries are sent to the optimizer (Section 4.3).  ... 
doi:10.1109/sc.2006.27 fatcat:yzq67pfs7za3ndj736tqdljapy

Data management and query---Estimating query result sizes for proxy caching in scientific database federations

Tanu Malik, Randal Burns, Nitesh V. Chawla, Alex Szalay
2006 Proceedings of the 2006 ACM/IEEE conference on Supercomputing - SC '06  
In a proxy cache for federations of scientific databases it is important to estimate the size of a query before making a caching decision.  ...  CAROT estimates query result sizes by learning the distribution of query results, not by examining or sampling data, but from observing workload.  ...  Acknowledgments The authors sincerely thank Jim Gray for his help with the selectivity estimation methods proposed for query optimizers.  ... 
doi:10.1145/1188455.1188562 dblp:conf/sc/MalikBCS06 fatcat:6avilhqbkzh6flkugosxtic57u

A Proposed Approach to Federated Query Optimization

Kavitha Juliet, Rajeswari R P
2017 IJARCCE  
The main objective of the paper is to give general framework for query optimization in federated database system and enhancing the global query optimization.  ...  Keeping such statistics up to date in the federation is troublesome due to local autonomy.This paper discuss query processing for a federated data base system.  ...  The federated query optimization is to find an execution plan for a user specified query that satisfies an optimal goal provided by the user.  ... 
doi:10.17148/ijarcce.2017.65139 fatcat:325ifwouxrem7evjfn2ve3fpwu

Federated Meta-Learning with Fast Convergence and Efficient Communication [article]

Fei Chen, Mi Luo, Zhenhua Dong, Zhenguo Li, Xiuqiang He
2019 arXiv   pre-print
Federated Averaging (FedAvg) which is a leading optimization algorithm in federated learning.  ...  In this work, we show that meta-learning is a natural choice to handle these issues, and propose a federated meta-learning framework FedMeta, where a parameterized algorithm (or meta-learner) is shared  ...  to new clients as a crucial property of federated learning.  ... 
arXiv:1802.07876v2 fatcat:gpi4ck56zbcnzopiraek5jabhe

Preface to the Special Issue on Data Management and Analysis Technique Supporting AI

Lei Chen, Department of Computer Science and Engineering, the Hong Kong University of Science and Technology, Hong Kong 999077, China, Hongzhi Wang, Yongxin Tong, Hong Gao
2021 International Journal of Software and Informatics  
achieve independence between data and application, task-oriented descriptive language and query optimization technology to improve task execution efficiency.  ...  training sub-process is customized and optimized according to the features of the task.  ...  ., professor of Beihang University, Ph.D. supervisor, senior member of CCF, is mainly engaged in the research on big data, databases, federated learning, spatiotemporal big data computing and crowd intelligence  ... 
doi:10.21655/ijsi.1673-7288.00244 fatcat:kgspofndnvacdafol3dm5wdmhu

Ephedra: Efficiently Combining RDF Data and Services Using SPARQL Federation [chapter]

Andriy Nikolov, Peter Haase, Johannes Trame, Artem Kozlov
2017 Communications in Computer and Information Science  
To address these needs, we present Ephedra: a SPARQL federation engine aimed at processing hybrid queries.  ...  Ephedra provides a flexible declarative mechanism for including hybrid services into a SPARQL federation and implements a number of static and runtime query optimization techniques for improving the hybrid  ...  With Ephedra we overcome these limitations: while adopting the SPARQL 1.1 federation mechanism, we broaden its usage to include custom services as data sources and optimize such hybrid queries to be executed  ... 
doi:10.1007/978-3-319-69548-8_17 fatcat:4xbpne6mwndyfilcranyllp2im

Swarm: A federated cloud framework for large-scale variant analysis

Amir Bahmani, Kyle Ferriter, Vandhana Krishnan, Arash Alavi, Amir Alavi, Philip S. Tsao, Michael P. Snyder, Cuiping Pan, Mihaela Pertea
2021 PLoS Computational Biology  
Compared to single-cloud platforms, the Swarm framework significantly reduced computational costs, run-time delays and risks of security breach and privacy violation.  ...  Here, we present Swarm, a framework for federated computation that promotes minimal data motion and facilitates crosstalk between genomic datasets stored on various cloud platforms.  ...  Acknowledgments We acknowledge the Stanford Genetics Bioinformatics Service Center (GBSC) for providing the gateway to GCP and AWS for this research.  ... 
doi:10.1371/journal.pcbi.1008977 pmid:33979321 fatcat:s62adyyabffwzkoczg7j36mr4i

The BigDAWG Polystore System

Jennie Duggan, Stan Zdonik, Aaron J. Elmore, Michael Stonebraker, Magda Balazinska, Bill Howe, Jeremy Kepner, Sam Madden, David Maier, Tim Mattson
2015 SIGMOD record  
Open questions in this space revolve around query optimization and the assignment of objects to storage engines.  ...  This paper presents a new view of federated databases to address the growing need for managing information that spans multiple data models.  ...  The next section indicates how BigDAWG learns a preference list for assigning simple and complex queries to engines.  ... 
doi:10.1145/2814710.2814713 fatcat:utkjzoijlvcxpm2rhykgfz5l4a

Search result diversification in resource selection for federated search

Dzung Hong, Luo Si
2013 Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval - SIGIR '13  
Prior research in resource selection for federated search mainly focused on selecting a small number of information sources that are most relevant to a user query.  ...  Moreover, this paper proposes a learning based approach to combine multiple resource selection algorithms for result diversification, which can further improve the performance.  ...  , and Bi denotes the same quantity with respect to the optimal ranking B.  ... 
doi:10.1145/2484028.2484091 dblp:conf/sigir/HongS13 fatcat:h2fb6wckznaanbfpihshke7hpe

Adaptive Join Operator for Federated Queries over Linked Data Endpoints [chapter]

Damla Oguz, Shaoyi Yin, Abdelkader Hameurlain, Belgin Ergenc, Oguz Dikenelli
2016 Lecture Notes in Computer Science  
Traditional static query optimization is not adequate for query federation over linked data endpoints due to unpredictable data arrival rates and missing statistics.  ...  In this paper, we propose an adaptive join operator for federated query processing which can change the join method during the execution.  ...  For this reason, we turn our attention to query federation. Query federation divides the query into subqueries and distributes them to the SPARQL endpoints of the selected data sources.  ... 
doi:10.1007/978-3-319-44039-2_19 fatcat:nctg65gs6fgztcjqwx4tjg25ie

Federated Data Science to Break Down Silos [Vision] [article]

Essam Mansour, Kavitha Srinivas, Katja Hose
2021 arXiv   pre-print
for efficient search and, in the ideal case, even allows for combining and defining pipelines across platforms in a federated manner.  ...  However, the few efforts that exist focus on the technical part on how to facilitate sharing, conversion, etc.  ...  Federated Graph Learning: KEK aims at developing a federated graph learning mechanism to learn graph representations (embeddings) across multiple DSKGs.  ... 
arXiv:2111.13186v1 fatcat:nod5jc3ppzashjqzir2l7yzm5q

Federated Learning for Ranking Browser History Suggestions [article]

Florian Hartmann, Sunah Suh, Arkadiusz Komarzewski, Tim D. Smith, Ilana Segall
2019 arXiv   pre-print
Our paper shows that Federated Learning can be used successfully to train models in privacy-respecting ways.  ...  Federated Learning is a new subfield of machine learning that allows fitting models without collecting the training data itself.  ...  ACKNOWLEDGMENTS The authors would like to extend their thanks to several people at Mozilla: Drew Willcoxon and Rob Helmer helped out with the Firefox client-side parts of the project.  ... 
arXiv:1911.11807v1 fatcat:hqmz7ut7j5av5bhpi5n7ujjrc4

An Optimization Framework for Merging Multiple Result Lists

Chia-Jung Lee, Qingyao Ai, W. Bruce Croft, Daniel Sheldon
2015 Proceedings of the 24th ACM International on Conference on Information and Knowledge Management - CIKM '15  
Federated web search, for instance, has become a common practice where a query is issued to different verticals and a single ranked list of blended results is created.  ...  Developing effective methods for fusing multiple ranked lists of documents is crucial to many applications.  ...  In web-scale federated search, it is beneficial to understand the right medium and identify appropriate verticals for a query.  ... 
doi:10.1145/2806416.2806489 dblp:conf/cikm/LeeACS15 fatcat:t63cqqlgy5hjtb7ow7zqbav2ci

BigDataGrapes D4.4 - Resource Optimization Methods and Algorithms

Vinicius Monteiro De Lira, Cristina Muntean, Franco Maria Nardini, Raffaele Perego, Nicola Tonellotto, Roberto Trani, Salvatore Trani
2020 Zenodo  
(BDG) platform to optimize computing resource management.  ...  This accompanying document for deliverable "D4.4 Resource Optimization Methods and Algorithms" describes new research tools to manage distributed big data platforms that will be used in the BigDataGrapes  ...  To this regard, the leading approach to learn machine learning models in this scenario is known as Federated Learning (FL).  ... 
doi:10.5281/zenodo.4546120 fatcat:yrnyjqbnsvfrxnzya3zvioswee
« Previous Showing results 1 — 15 out of 24,706 results