A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2018; you can also visit the original URL.
The file type is application/pdf.
Filters
The Linked Data Benchmark Council Project
2013
Datenbank-Spektrum
Preserved Fulltext
recently launched a European Commision sponsored project that will offer the first comprehensive set of open and vendor-independent benchmarks for RDF and Graph technologies. ...
Despite the fast growth and increasing popularity, the broad field of RDF and Graph database systems lacks an independent authority for developing benchmarks, and for neutrally assessing benchmark results ...
We hope that, similarly to relational databases, adequate benchmarking will advance research in many aspects of RDF and Graph data management. ...
doi:10.1007/s13222-013-0125-y
fatcat:qmqlsg23djf5dn24eavhffhuou
Benchmarks for Corruption Invariant Person Re-identification
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Note that this fulltext copy is not of the "primary" version of this work. The version it corresponds to is:
2021 pre-print arXiv:2111.00880v1 fatcat:r2muls24yjdytmlheupwb2ghqq
After reproducing and examining the robustness performance of 21 recent ReID methods, we have some observations: 1) transformer-based models are more robust towards corrupted images, compared with CNN-based ...
In this work, we comprehensively establish six ReID benchmarks for learning corruption invariant representation. ...
Acknowledgments and disclosure of funding. This work is supported by the National Natural Science Foundation of China under Grant No. 61972188 and No.62122035. ...
arXiv:2111.00880v2
fatcat:7tvgf5sgwjbitmosfgfhs7bcmi
AIBench Scenario: Scenario-distilling AI Benchmarking
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Modern real-world application scenarios like Internet services consist of a diversity of AI and non-AI modules with huge code sizes and long and complicated execution paths, which raises serious benchmarking ...
We formalize a real-world application scenario as a Directed Acyclic Graph-based model and propose the rules to distill it into a permutation of essential AI and non-AI tasks, which we call a scenario ...
for our communities towards developing an open and mature research field. ...
arXiv:2005.03459v4
fatcat:ino6quufafetngeahcrzvirnbi
SparkBench – A Spark Performance Testing Suite
[chapter]
2016
Lecture Notes in Computer Science
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2015; you can also visit the original URL.
The file type is application/pdf.
Spark has emerged as an easy to use, scalable, robust and fast system for analytics with a rapidly growing and vibrant community of users and contributors. ...
Recognizing the need for such comprehensive and agile testing, this paper proposes going beyond existing performance tests for Spark and creating an expanded Spark performance testing suite. ...
Acknowledgements The authors would like to acknowledge all those who contributed with suggestions, ideas and provided valuable feedback during earlier drafts of this document. ...
doi:10.1007/978-3-319-31409-9_3
fatcat:koanet7fdfdfnegkhujk3pj27q
Towards Universal Backward-Compatible Representation Learning
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Other Versions
open-set scenarios. ...
Conventional model upgrades for visual search systems require offline refresh of gallery features by feeding gallery images into new models (dubbed as "backfill"), which is time-consuming and expensive ...
It is notable that the data split for model upgrades in real-world applications is complex and unpredictable, that is, both close-set and open-set scenarios should be considered. ...
arXiv:2203.01583v2
fatcat:wilqpv2pozflbbicips33nulqm
Issues in big data testing and benchmarking
2013
Proceedings of the Sixth International Workshop on Testing Database Systems - DBTest '13
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2013; you can also visit the original URL.
The file type is application/pdf.
from real world data. ...
In this paper, we describe initial solutions and challenges with respect to big data generation, methods for creating realistic, privacy-aware, and arbitrarily scalable data sets, workloads, and benchmarks ...
sets and queries from well-known benchmarks, like TPC-H, TPC-C [TPC13], and XMLGen [XML13]. ...
doi:10.1145/2479440.2482677
dblp:conf/sigmod/AlexandrovBM13
fatcat:hpd7rq2mm5emvlhk77rdvccub4
Towards Real-World Applications of ServiceX, an Analysis Data Transformation System
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
ServiceX is a software R&D project in the area of Data Organization, Management and Access of the IRIS- HEP to investigate new computational models for the HL- LHC era. ...
It is capable of retrieving data from grid sites, on-the-fly data transformation, and delivering user-selected data in a variety of different formats. ...
Carlo samples for signal and background and data samples, sources of systematic uncertainties, details of fit model, parameter of interest, and other general settings. ...
arXiv:2107.01789v1
fatcat:52i4wkvklvchzj224rg2rjk2k4
Towards Real-World Applications of ServiceX, an Analysis Data Transformation System
2021
EPJ Web of Conferences
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
ServiceX is a software R&D project in the area of Data Organization, Management and Access of the IRIS-HEP to investigate new computational models for the HLLHC era. ...
It is capable of retrieving data from grid sites, on-the-fly data transformation, and delivering user-selected data in a variety of different formats. ...
Carlo samples for signal and background and data samples, sources of systematic uncertainties, details of fit model, parameter of interest, and other general settings. ...
doi:10.1051/epjconf/202125102053
fatcat:5kh74g72ondgrduscn43gbigxq
Where is Linked Data in Question Answering over Linked Data?
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
To this end, we propose the creation of new evaluation settings to leverage the advantages of the Semantic Web to achieve AI-complete question answering. ...
We argue that "Question Answering with Knowledge Base" and "Question Answering over Linked Data" are currently two instances of the same problem, despite one explicitly declares to deal with Linked Data ...
Acknowledgments We thank Dennis Diefenbach for his kind suggestions and feedback. ...
arXiv:2005.03640v1
fatcat:i7gxlwcrlbadxetjdjkmtzaltq
Open Logo Detection Challenge
[article]
2018
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
We further create an open logo detection benchmark, called OpenLogo,to promote the investigation of this new challenge. ...
Existing benchmarks are thus unable to evaluate the true performance of a logo detection method in realistic and open deployments. ...
Project on Developing and Commercialising Intelligent Video Analytics Solutions for Public Safety (98111-571149). ...
arXiv:1807.01964v3
fatcat:ih6tgdepbjcctfgj4hrvppueg4
AIBench: An Industry Standard Internet Service AI Benchmark Suite
[article]
2019
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
On the basis of the AIBench framework, abstracting the real-world data sets and workloads from one of the top e-commerce providers, we design and implement the first end-to-end Internet service AI benchmark ...
The diversity of these modules and complexity of execution paths, the massive scale and complex hierarchy of datacenter infrastructure, the confidential issues of data sets and workloads pose great challenges ...
, real-world data sets and software stacks. ...
arXiv:1908.08998v2
fatcat:5ngwtjnk7zhrrmc4oedoje633e
BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Other Versions
To address this, and to facilitate researchers to broadly evaluate the effectiveness of their models, we introduce Benchmarking-IR (BEIR), a robust and heterogeneous evaluation benchmark for information ...
Existing neural information retrieval (IR) models have often been studied in homogeneous and narrow settings, which has considerably limited insights into their out-of-distribution (OOD) generalization ...
This setting hinders for investigation of model generalization across diverse set of domains and task types. ...
arXiv:2104.08663v4
fatcat:fow5uqghbzggjclobtv7bpjtaa
Big Data Benchmark Compendium
[chapter]
2016
Lecture Notes in Computer Science
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
Therefore, the traditional way of specifying a standardized benchmark with pre-defined workloads, which have been in use for years in the transaction and analytical processing systems, is not trivial to ...
As these benchmarks are emphasizing different aspects of Big Data and, in many cases, covering different technical platforms and uses cases, it is extremely difficult to keep up with the pace of benchmark ...
Acknowledgment This research has been supported by the Research Group of the Standard Performance Evaluation Corporation (SPEC). ...
doi:10.1007/978-3-319-31409-9_9
fatcat:n7lwtxainnblpf2xp4c5o2eynq
Assessment and Benchmarking of Spatially Enabled RDF Stores for the Next Generation of Spatial Data Infrastructure
2019
ISPRS International Journal of Geo-Information
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Geospatial information is indispensable for various real-world applications and is thus a prominent part of today's data science landscape. ...
In this scenario, we used ICOS Carbon Portal's metadata—a real-world Earth Science linked data infrastructure. ...
Funding: The work was supported by Lund University and China Scholarship Council.
Conflicts of Interest: The authors declare no conflict of interest. ...
doi:10.3390/ijgi8070310
fatcat:dczpbeeyhndbboxr2lxuhojj6m
Benchmarking Knowledge Graphs on the Web
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
The growing interest in making use of Knowledge Graphs for developing explainable artificial intelligence, there is an increasing need for a comparable and repeatable comparison of the performance of Knowledge ...
This paper gives an overview of benchmarks used to evaluate systems that process Knowledge Graphs. ...
Real-data benchmarks make use of real-world datasets and queries from real user query logs for benchmarking. ...
arXiv:2002.06039v1
fatcat:tc25o6rplbekndjng4cqh2oxoe
« Previous
Showing results 1 — 15 out of 19,300 results