Filters








19,300 Hits in 6.8 sec

The Linked Data Benchmark Council Project

Peter Boncz, Irini Fundulaki, Andrey Gubichev, Josep Larriba-Pey, Thomas Neumann
2013 Datenbank-Spektrum  
recently launched a European Commision sponsored project that will offer the first comprehensive set of open and vendor-independent benchmarks for RDF and Graph technologies.  ...  Despite the fast growth and increasing popularity, the broad field of RDF and Graph database systems lacks an independent authority for developing benchmarks, and for neutrally assessing benchmark results  ...  We hope that, similarly to relational databases, adequate benchmarking will advance research in many aspects of RDF and Graph data management.  ... 
doi:10.1007/s13222-013-0125-y fatcat:qmqlsg23djf5dn24eavhffhuou

Benchmarks for Corruption Invariant Person Re-identification [article]

Minghui Chen, Zhiqiang Wang, Feng Zheng
2022 arXiv   pre-print
After reproducing and examining the robustness performance of 21 recent ReID methods, we have some observations: 1) transformer-based models are more robust towards corrupted images, compared with CNN-based  ...  In this work, we comprehensively establish six ReID benchmarks for learning corruption invariant representation.  ...  Acknowledgments and disclosure of funding. This work is supported by the National Natural Science Foundation of China under Grant No. 61972188 and No.62122035.  ... 
arXiv:2111.00880v2 fatcat:7tvgf5sgwjbitmosfgfhs7bcmi

AIBench Scenario: Scenario-distilling AI Benchmarking [article]

Wanling Gao, Fei Tang, Jianfeng Zhan, Xu Wen, Lei Wang, Zheng Cao, Chuanxin Lan, Chunjie Luo, Xiaoli Liu, Zihan Jiang
2021 arXiv   pre-print
Modern real-world application scenarios like Internet services consist of a diversity of AI and non-AI modules with huge code sizes and long and complicated execution paths, which raises serious benchmarking  ...  We formalize a real-world application scenario as a Directed Acyclic Graph-based model and propose the rules to distill it into a permutation of essential AI and non-AI tasks, which we call a scenario  ...  for our communities towards developing an open and mature research field.  ... 
arXiv:2005.03459v4 fatcat:ino6quufafetngeahcrzvirnbi

SparkBench – A Spark Performance Testing Suite [chapter]

Dakshi Agrawal, Ali Butt, Kshitij Doshi, Josep-L. Larriba-Pey, Min Li, Frederick R Reiss, Francois Raab, Berni Schiefer, Toyotaro Suzumura, Yinglong Xia
2016 Lecture Notes in Computer Science  
Spark has emerged as an easy to use, scalable, robust and fast system for analytics with a rapidly growing and vibrant community of users and contributors.  ...  Recognizing the need for such comprehensive and agile testing, this paper proposes going beyond existing performance tests for Spark and creating an expanded Spark performance testing suite.  ...  Acknowledgements The authors would like to acknowledge all those who contributed with suggestions, ideas and provided valuable feedback during earlier drafts of this document.  ... 
doi:10.1007/978-3-319-31409-9_3 fatcat:koanet7fdfdfnegkhujk3pj27q

Towards Universal Backward-Compatible Representation Learning [article]

Binjie Zhang, Yixiao Ge, Yantao Shen, Shupeng Su, Fanzi Wu, Chun Yuan, Xuyuan Xu, Yexin Wang, Ying Shan
2022 arXiv   pre-print
open-set scenarios.  ...  Conventional model upgrades for visual search systems require offline refresh of gallery features by feeding gallery images into new models (dubbed as "backfill"), which is time-consuming and expensive  ...  It is notable that the data split for model upgrades in real-world applications is complex and unpredictable, that is, both close-set and open-set scenarios should be considered.  ... 
arXiv:2203.01583v2 fatcat:wilqpv2pozflbbicips33nulqm

Issues in big data testing and benchmarking

Alexander Alexandrov, Christoph Brücke, Volker Markl
2013 Proceedings of the Sixth International Workshop on Testing Database Systems - DBTest '13  
from real world data.  ...  In this paper, we describe initial solutions and challenges with respect to big data generation, methods for creating realistic, privacy-aware, and arbitrarily scalable data sets, workloads, and benchmarks  ...  sets and queries from well-known benchmarks, like TPC-H, TPC-C [TPC13], and XMLGen [XML13].  ... 
doi:10.1145/2479440.2482677 dblp:conf/sigmod/AlexandrovBM13 fatcat:hpd7rq2mm5emvlhk77rdvccub4

Towards Real-World Applications of ServiceX, an Analysis Data Transformation System [article]

KyungEon Choi, Andrew Eckart, Ben Galewsky, Robert Gardner, Mark S. Neubauer, Peter Onyisi, Mason Proffitt, Ilija Vukotic, Gordon T. Watts
2021 arXiv   pre-print
ServiceX is a software R&D project in the area of Data Organization, Management and Access of the IRIS- HEP to investigate new computational models for the HL- LHC era.  ...  It is capable of retrieving data from grid sites, on-the-fly data transformation, and delivering user-selected data in a variety of different formats.  ...  Carlo samples for signal and background and data samples, sources of systematic uncertainties, details of fit model, parameter of interest, and other general settings.  ... 
arXiv:2107.01789v1 fatcat:52i4wkvklvchzj224rg2rjk2k4

Towards Real-World Applications of ServiceX, an Analysis Data Transformation System

K. Choi, A. Eckart, B. Galewsky, R. Gardner, M. Neubauer, P. Onyisi, M. Proffitt, I. Vukotic, G. Watts, C. Biscarat, S. Campana, B. Hegner (+3 others)
2021 EPJ Web of Conferences  
ServiceX is a software R&D project in the area of Data Organization, Management and Access of the IRIS-HEP to investigate new computational models for the HLLHC era.  ...  It is capable of retrieving data from grid sites, on-the-fly data transformation, and delivering user-selected data in a variety of different formats.  ...  Carlo samples for signal and background and data samples, sources of systematic uncertainties, details of fit model, parameter of interest, and other general settings.  ... 
doi:10.1051/epjconf/202125102053 fatcat:5kh74g72ondgrduscn43gbigxq

Where is Linked Data in Question Answering over Linked Data? [article]

Tommaso Soru, Edgard Marx, André Valdestilhas, Diego Moussallem, Gustavo Publio, Muhammad Saleem
2020 arXiv   pre-print
To this end, we propose the creation of new evaluation settings to leverage the advantages of the Semantic Web to achieve AI-complete question answering.  ...  We argue that "Question Answering with Knowledge Base" and "Question Answering over Linked Data" are currently two instances of the same problem, despite one explicitly declares to deal with Linked Data  ...  Acknowledgments We thank Dennis Diefenbach for his kind suggestions and feedback.  ... 
arXiv:2005.03640v1 fatcat:i7gxlwcrlbadxetjdjkmtzaltq

Open Logo Detection Challenge [article]

Hang Su, Xiatian Zhu, Shaogang Gong
2018 arXiv   pre-print
We further create an open logo detection benchmark, called OpenLogo,to promote the investigation of this new challenge.  ...  Existing benchmarks are thus unable to evaluate the true performance of a logo detection method in realistic and open deployments.  ...  Project on Developing and Commercialising Intelligent Video Analytics Solutions for Public Safety (98111-571149).  ... 
arXiv:1807.01964v3 fatcat:ih6tgdepbjcctfgj4hrvppueg4

AIBench: An Industry Standard Internet Service AI Benchmark Suite [article]

Wanling Gao, Fei Tang, Lei Wang, Jianfeng Zhan, Chunxin Lan, Chunjie Luo, Yunyou Huang, Chen Zheng, Jiahui Dai, Zheng Cao, Daoyi Zheng, Haoning Tang (+12 others)
2019 arXiv   pre-print
On the basis of the AIBench framework, abstracting the real-world data sets and workloads from one of the top e-commerce providers, we design and implement the first end-to-end Internet service AI benchmark  ...  The diversity of these modules and complexity of execution paths, the massive scale and complex hierarchy of datacenter infrastructure, the confidential issues of data sets and workloads pose great challenges  ...  , real-world data sets and software stacks.  ... 
arXiv:1908.08998v2 fatcat:5ngwtjnk7zhrrmc4oedoje633e

BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models [article]

Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, Iryna Gurevych
2021 arXiv   pre-print
To address this, and to facilitate researchers to broadly evaluate the effectiveness of their models, we introduce Benchmarking-IR (BEIR), a robust and heterogeneous evaluation benchmark for information  ...  Existing neural information retrieval (IR) models have often been studied in homogeneous and narrow settings, which has considerably limited insights into their out-of-distribution (OOD) generalization  ...  This setting hinders for investigation of model generalization across diverse set of domains and task types.  ... 
arXiv:2104.08663v4 fatcat:fow5uqghbzggjclobtv7bpjtaa

Big Data Benchmark Compendium [chapter]

Todor Ivanov, Tilmann Rabl, Meikel Poess, Anna Queralt, John Poelman, Nicolas Poggi, Jeffrey Buell
2016 Lecture Notes in Computer Science  
Therefore, the traditional way of specifying a standardized benchmark with pre-defined workloads, which have been in use for years in the transaction and analytical processing systems, is not trivial to  ...  As these benchmarks are emphasizing different aspects of Big Data and, in many cases, covering different technical platforms and uses cases, it is extremely difficult to keep up with the pace of benchmark  ...  Acknowledgment This research has been supported by the Research Group of the Standard Performance Evaluation Corporation (SPEC).  ... 
doi:10.1007/978-3-319-31409-9_9 fatcat:n7lwtxainnblpf2xp4c5o2eynq

Assessment and Benchmarking of Spatially Enabled RDF Stores for the Next Generation of Spatial Data Infrastructure

Weiming Huang, Syed Amir Raza, Oleg Mirzov, Lars Harrie
2019 ISPRS International Journal of Geo-Information  
Geospatial information is indispensable for various real-world applications and is thus a prominent part of today's data science landscape.  ...  In this scenario, we used ICOS Carbon Portal's metadata—a real-world Earth Science linked data infrastructure.  ...  Funding: The work was supported by Lund University and China Scholarship Council. Conflicts of Interest: The authors declare no conflict of interest.  ... 
doi:10.3390/ijgi8070310 fatcat:dczpbeeyhndbboxr2lxuhojj6m

Benchmarking Knowledge Graphs on the Web [article]

Michael Röder and Mohamed Ahmed Sherif and Muhammad Saleem and Felix Conrads and Axel-Cyrille Ngonga Ngomo
2020 arXiv   pre-print
The growing interest in making use of Knowledge Graphs for developing explainable artificial intelligence, there is an increasing need for a comparable and repeatable comparison of the performance of Knowledge  ...  This paper gives an overview of benchmarks used to evaluate systems that process Knowledge Graphs.  ...  Real-data benchmarks make use of real-world datasets and queries from real user query logs for benchmarking.  ... 
arXiv:2002.06039v1 fatcat:tc25o6rplbekndjng4cqh2oxoe
« Previous Showing results 1 — 15 out of 19,300 results