A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is
Serving deep neural networks in latency critical interactive settings often requires GPU acceleration. However, the small batch sizes typical in online inference results in poor GPU utilization, a potential performance gap which GPU resource sharing can address. In this paper, we explore several techniques to leverage both temporal and spatial multiplexing to improve GPU utilization for deep learning inference workloads. We evaluate the performance trade-offs of each approach with respect toarXiv:1901.00041v1 fatcat:hzxlziaftvbypnwtls5a4uqk6e
more »... ource-efficiency, latency predictability, and isolation when compared with conventional batched inference. Our experimental analysis suggests up to a 5x potential for improved utilization through the exploration of more advanced spatial and temporal multiplexing strategies. Our preliminary prototype of a dynamic space-time scheduler demonstrates a 3.23x floating-point throughput increase over space-only multiplexing and a 7.73x increase over time-only multiplexing for convolutions, while also providing better isolation and latency predictability.
ACKNOWLEDGMENTS We thank Hari Subbaraj and Rehan Sohail Durrani who helped profile kernels as well as Steven Hand, Koushik Sen, Eyal Sela, Zongheng Yang, Anjali Shankar and Daniel Crankshaw for their insightful ...arXiv:1901.10008v2 fatcat:xie3vplzwvbotginlsywecppae
, Durrani & Khan, 2017). ... indispensable means which should be used by organizations to realize the different harms to the local environment which further leads to the physical and mental health of the individuals in the community (Sohail ...doi:10.46377/dilemas.v29i1.1862 fatcat:bulxwrv7lzb6bdmpgam4uzy5p4
Sohail Durrani. ... Acknowledgments We would like to acknowledge those who have made significant contributions to the MODIN codebase: we thank Omkar Salpekar, Eran Avidan, Kunal Gosar, GitHub user ipacheco-uy, Alex Wu, and Rehan ...arXiv:2001.00888v4 fatcat:ewwmj6dzubaijjjc4mwqfvarrq
REHAN GUL, DR. HUMAYUN BASHIR, BASHIR AHMED, M. MANAN BHATTI, DEPARTMENT OF NUCLEAR MEDICINE, SHAUKAT KHANUM MEMORIAL CANCER HOSPITAL AND RESEARCH CENTRE, LAHORE, PAKISTAN. ... order to determine the success of this procedure in terms of graft integrity, recurrence and use of the arm in adulthood. 101-P A STUDY OF PERINEURAL INVASION IN ORAL SQUAMOUS CELL CARCINOMA ZUBAIR DURRANI ...doi:10.37029/jcas.v4i4.216 fatcat:dmrlbqxxoja3hd2ft6xekvtryu
It was produced and directed by Javed Jabbar with music by Sohail Rana, starring Usman Peerzada, Zahoor Ahmad, Subhani Bayounus and Raja Film Aina, released in March 1977, had Nadeem, Shabnam, Rehan, Qavi ... The movie was produced for Punjab Pictures by Ijaz Durrani, who also starred along with star cast viz., Shabnam, Husna, Rahman, Saqi. ...doi:10.13140/rg.2.2.33901.74720 fatcat:h3k3gok64vda7gcl4yn4ghrmoq