Presentation
Flex-SFU: Accelerating DNN Activation Functions by Non-Uniform Piecewise Approximation
DescriptionModern DNN workloads increasingly rely on computationally complex activation functions, posing challenges to current accelerators optimized for convolutions and matrix-matrix-multiplications. We present Flex-SFU, a lightweight hardware-accelerator for activation functions implementing non-uniform piecewise-interpolation, supporting multiple data-formats using binary-tree comparisons for LUT-address-decoding. Thanks to non-uniform interpolation, floating-point support, and an SGD-based optimization algorithm with heuristics optimizing mean-squared-error (MSE), Flex-SFU achieves on average 32x better MSE compared to previous PWL-approaches. The end-to-end-performance of state-of-the-art networks on AI hardware-accelerators improves by 22.8\%, minimally (not) impacting the model's accuracy when using 16 (32) segments, while introducing an area/power overhead of 3.5\%/0.7\% relative to our vector-processor-baseline.
Event Type
Research Manuscript
TimeWednesday, July 12th3:40pm - 3:55pm PDT
Location3010, 3rd Floor
Design
AI/ML Architecture Design