1 Hit in 5.1 sec

The OoO VLIW JIT Compiler for GPU Inference [article]

Paras Jain, Xiangxi Mo, Ajay Jain, Alexey Tumanov, Joseph E. Gonzalez, Ion Stoica
2019 arXiv   pre-print
We propose a VLIW-inspired Out-of-Order (OoO) Just-in-Time (JIT) compiler that coalesces and reorders execution kernels at runtime for throughput-optimal device utilization while satisfying latency SLOs  ...  As ML inference is increasingly time-bounded by tight latency SLOs, increasing data parallelism is not an option. The need for better efficiency motivates GPU multiplexing.  ...  We propose an Out-of-Order (OoO) Just-in-Time (JIT) GPU kernel VLIW-inspired compiler for DNN inference ( Figure 1 ).  ... 
arXiv:1901.10008v2 fatcat:xie3vplzwvbotginlsywecppae