Presentation
OmniBoost: Boosting Throughput of Heterogeneous Embedded Devices under Multi-DNN Workload
DescriptionScheduling multiple DNN applications introduces new challenges regarding workload distribution. Equipped with a diverge set of accelerators, newer embedded system present architectural heterogeneity, which current run-time controllers are unable to fully utilize. To enable high throughput in multi-DNN workloads, such a controller is ought to explore a vast solution space to exploit the underlying heterogeneity. In this paper, we propose OmniBoost, a lightweight and extensible multi-DNN manager for heterogeneous embedded devices. We leverage stochastic space exploration and we combine it with a highly accurate performance estimator to observe a x4.6 average throughput boost compared to other state-of-the-art methods.
Event Type
Research Manuscript
TimeTuesday, July 11th1:40pm - 1:55pm PDT
Location3008, 3rd Floor
Embedded Systems
Embedded System Design Methodologies