Filters








23 Hits in 7.8 sec

MLPerf Mobile Inference Benchmark [article]

Vijay Janapa Reddi, David Kanter, Peter Mattson, Jared Duke, Thai Nguyen, Ramesh Chukka, Ken Shiring, Koan-Sin Tan, Mark Charlebois, William Chou, Mostafa El-Khamy, Jungwook Hong (+15 others)
2022 arXiv   pre-print
MLPerf Mobile will evolve and serve as an open-source community framework to guide research and innovation for mobile AI.  ...  This paper presents the first industry-standard open-source machine learning (ML) benchmark to allow perfor mance and accuracy evaluation of mobile devices with different AI chips and software stacks.  ...  INSIGHTS FROM BENCHMARK RESULTS To understand what is to be gained from benchmarking, we dissect the first two rounds of MLPerf Mobile submissions.  ... 
arXiv:2012.02328v4 fatcat:uu7p5nk2pzhepnayhwwtspklwa

MLPerf Mobile Inference Benchmark: An Industry-Standard Open-Source Machine Learning Benchmark for On-Device AI

Vijay Janapa Reddi, David Kanter, Peter Mattson, Jared Duke, Thai Nguyen, Ramesh Chukka, Kenneth Shiring, Koan-Sin Tan, Mark Charlebois, William Chou, Mostafa El-Khamy, Jungwook Hong (+15 others)
2022 Conference on Machine Learning and Systems  
MLPerf Mobile will evolve and serve as an open-source community framework to guide research and innovation for mobile AI.  ...  This paper presents the first industry-standard open-source machine learning (ML) benchmark to allow performance and accuracy evaluation of mobile devices with different AI chips and software stacks.  ...  INSIGHTS FROM BENCHMARK RESULTS To understand what is to be gained from benchmarking, we dissect the first two rounds of MLPerf Mobile submissions.  ... 
dblp:conf/mlsys/ReddiKMDNCSTCCE22 fatcat:ycl3tay65za47knjj37odpopia

MLPerf Inference Benchmark [article]

Vijay Janapa Reddi, Christine Cheng, David Kanter, Peter Mattson, Guenther Schmuelling, Carole-Jean Wu, Brian Anderson, Maximilien Breughe, Mark Charlebois, William Chou, Ramesh Chukka, Cody Coleman (+35 others)
2020 arXiv   pre-print
There is a clear need for industry-wide standard ML benchmarking and evaluation criteria. MLPerf Inference answers that call.  ...  Machine-learning (ML) hardware and software system demand is burgeoning. Driven by ML applications, the number of different ML inference systems has exploded.  ...  ACKNOWLEDGEMENTS MLPerf Inference is the work of many individuals from multiple organizations.  ... 
arXiv:1911.02549v2 fatcat:jewandlmivctjb7wywadxuuqju

HPC AI500: The Methodology, Tools, Roofline Performance Models, and Metrics for Benchmarking HPC AI Systems [article]

Zihan Jiang, Lei Wang, Xingwang Xiong, Wanling Gao, Chunjie Luo, Fei Tang, Chuanxin Lan, Hongxiao Li, Jianfeng Zhan
2020 arXiv   pre-print
The recent years witness a trend of applying large-scale distributed deep learning in both business and scientific computing areas, whose goal is to speed up the training time to achieve a state-of-the-art  ...  We abstract the HPC AI system into nine independent layers, and present explicit benchmarking rules and procedures to assure equivalence of each layer, repeatability, and replicability.  ...  We also thank Shaomeng Cao, Xuhui Shao, Yongheng Liu, Changsong Liu, and Jingfei Qiu for technical support in using those systems.  ... 
arXiv:2007.00279v1 fatcat:mweupqwxffapxfid3kkdnvnroy

MARVEL - D5.2: Technical evaluation and progress against benchmarks – initial version

Toni Heittola, Tuomas Virtanen
2022 Zenodo  
The purpose of this deliverable is to describe in detail the technical evaluation and progress against benchmarks.  ...  In addition to this, the contribution to MARVEL KPIs per component is described and expected future results are discussed.  ...  Of especial interest are benchmarks to assess AI and federated learning, such as AIBench, MLPerf, or LEAF 12 , as well as benchmarking approaches followed in AIBench and MLPerf, some of them dealing with  ... 
doi:10.5281/zenodo.6322699 fatcat:d5lpwby5szg4fih77sp5rjoyae

Evaluation of Optimized CNNs on Heterogeneous Accelerators using a Novel Benchmarking Approach

Michaela Blott, Nicholas Fraser, Giulio Gambardella, Lisa Halder, Johannes Kath, Zachary Neveu, Yaman Umuroglu, Alina Vasilciuc, Miriam Leeser, Linda Doyle
2020 IEEE transactions on computers  
Given the broad selection of AI accelerators, it is not obvious which approach benefits from which optimization most.  ...  Our findings show that channel pruning is most effective and works across most hardware platforms, with speedups directly correlated to the reduction in compute load, while FPGAs benefit the most from  ...  MLPerf [42] is the most promising industry-wide effort in establishing a fair benchmark for CNN inference and training.  ... 
doi:10.1109/tc.2020.3022318 fatcat:qbzrs75ierayvovbibwm2ezcge

Metadata Representations for Queryable ML Model Zoos [article]

Ziyu Li, Rihan Hai, Alessandro Bozzon, Asterios Katsifodimos
2022 arXiv   pre-print
The metatada is currently not standardised; its expressivity is limited; and there is no interoperable way to store and query it.  ...  In this paper, we advocate for standardized ML model meta-data representation and management, proposing a toolkit supported to help practitioners manage and query that metadata.  ...  ., 2019) , but such information is mostly for human consumption, making it hard for automatic extension or management.  ... 
arXiv:2207.09315v1 fatcat:okbik52a2na57oiia7wfpta7ki

Compute and Energy Consumption Trends in Deep Learning Inference [article]

Radosvet Desislavov, Fernando Martínez-Plumed, José Hernández-Orallo
2021 arXiv   pre-print
The progress of some AI paradigms such as deep learning is said to be linked to an exponential growth in the number of parameters.  ...  The only caveat is, yet again, the multiplicative factor, as future AI increases penetration and becomes more pervasive.  ...  From the AI research community, we have more to say and do about the former. Accordingly, more effort is needed, within AI, to better account for the internalities, as we do in this paper.  ... 
arXiv:2109.05472v1 fatcat:v5dbktk7xzefjnsstyneb2vyqi

The AI Index 2021 Annual Report [article]

Daniel Zhang, Saurabh Mishra, Erik Brynjolfsson, John Etchemendy, Deep Ganguli, Barbara Grosz, Terah Lyons, James Manyika, Juan Carlos Niebles, Michael Sellitto, Yoav Shoham, Jack Clark (+1 others)
2021 arXiv   pre-print
Its mission is to provide unbiased, rigorously vetted, and globally sourced data for policymakers, researchers, executives, journalists, and the general public to develop intuitions about the complex field  ...  The report aims to be the most credible and authoritative source for data and insights about AI in the world.  ...  The terms searched for were based on the issues exposed and identified in papers below, and also on the topics called for discussion in the First AAAI/ACM Conference on AI, Ethics, and Society.  ... 
arXiv:2103.06312v1 fatcat:52qwvzv7jndxzaagyiro6koyza

Leveraging the Openness and Modularity of RISC-V in Space

Stefano Di Mascio, Alessandra Menicucci, Eberhard Gill, Gianluca Furano, Claudio Monteleone
2019 Journal of Aerospace Information Systems  
Future space systems could benefit from many of those developments, and this work identifies and highlights what is still missing to satisfy the specific needs of processors for space, especially in terms  ...  "MIPS" is an acronym universally accepted to indicate both the "Microprocessor Without Interlocked Pipelined Stages" ISA and the millions of instructions per second metric to measure processing speed.  ...  Acknowledgments This work was supported by the European Space Agency under the NPI Program, Cobham Gaisler AB, and Delft University of Technology.  ... 
doi:10.2514/1.i010735 fatcat:b4ckmbr2uvhvzi57ltesqyiokm

Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims [article]

Miles Brundage, Shahar Avin, Jasmine Wang, Haydn Belfield, Gretchen Krueger, Gillian Hadfield, Heidy Khlaaf, Jingying Yang, Helen Toner, Ruth Fong, Tegan Maharaj, Pang Wei Koh (+47 others)
2020 arXiv   pre-print
evidence about the safety, security, fairness, and privacy protection of AI systems.  ...  This report suggests various steps that different stakeholders can take to improve the verifiability of claims made about AI systems and their associated development processes, with a focus on providing  ...  A user-centric design perspective points to many questions such as: What do people need to know about an Oak app when making a decision about whether to use it?  ... 
arXiv:2004.07213v2 fatcat:4xii6rzlyffjnj3nlb47tbqi4y

A Roadmap for Big Model [article]

Sha Yuan, Hanyu Zhao, Shuai Zhao, Jiahong Leng, Yangxiao Liang, Xiaozhi Wang, Jifan Yu, Xin Lv, Zhou Shao, Jiaao He, Yankai Lin, Xu Han (+88 others)
2022 arXiv   pre-print
At present, there is a lack of research work that sorts out the overall progress of BMs and guides the follow-up research.  ...  and Application.  ...  What is more, lower precision data can be used to model inference by quantization techniques. Overall, AI computing does not require high precision and its calculation on the node is simple.  ... 
arXiv:2203.14101v4 fatcat:rdikzudoezak5b36cf6hhne5u4

State of AI Ethics Report (Volume 6, February 2022) [article]

Abhishek Gupta
2022 arXiv   pre-print
It is a resource for researchers and practitioners alike in the field to set their research and development agendas to make contributions to the field of AI ethics.  ...  The report is a comprehensive overview of what the key issues in the field of AI ethics were in 2021, what trends are emergent, what gaps exist, and a peek into what to expect from the field of AI ethics  ...  AI industry, obsessed with speed, is loathe to consider the energy cost in latest MLPerf benchmark [Original article by ZDNet ] What happened : In the latest MLPerf benchmark results, a benchmark that  ... 
arXiv:2202.07435v1 fatcat:wbalu3j3ynelfim7eqvxcpwu3q

Understanding Time Variations of DNN Inference in Autonomous Driving [article]

Liangkai Liu, Yanzhi Wang, Weisong Shi
2022 arXiv   pre-print
In safety-critical systems like autonomous driving, executing tasks like sensing and perception in real-time is vital to the vehicle's safety, which requires the application's execution time to be predictable  ...  However, non-negligible time variations are observed in DNN inference. Current DNN inference studies either ignore the time variation issue or rely on the scheduler to handle it.  ...  • How vital is timing variation for autonomous driving, and what is the state-of-the-art? • What are the potential issues that affected the time variations in DNN inference? A.  ... 
arXiv:2209.05487v1 fatcat:kpaknrfxnbazrn4whtucrdhhmq

Distributed Training of Deep Learning Models: A Taxonomic Perspective

Matthias Langer, Zhen He, Wenny Rahayu, Yanbo Xue
2020 IEEE Transactions on Parallel and Distributed Systems  
Comparing DDLS side-by-side is difficult due to their extensive feature lists and architectural deviations.  ...  deep learning models and how such workloads can be distributed in a cluster to achieve collaborative model training.  ...  Therefore, the collection and quantitative study of the performance of DDLS using standardized AI benchmarks is becoming increasingly important and can provide guidance regarding what configurations work  ... 
doi:10.1109/tpds.2020.3003307 fatcat:awfmstytq5d5jlnavlvzezklgi
« Previous Showing results 1 — 15 out of 23 results