Presentation
TF-MVP: Novel Sparsity-Aware Transformer Accelerator with Mixed-Length Vector Pruning
DescriptionWe present the energy-efficient TF-MVP architecture, a sparsity-aware transformer accelerator by introducing novel algorithm-hardware co-optimization approaches.
From the fine-grained pruning map, the direction score is developed to analyze the pruning patterns quantitatively for the first time, indicating the major pruning direction and size of each layer.
Then, the mixed-length vector pruning is applied to generate the hardware-friendly pruned-transformer model, which is fully supported by the proposed TF-MVP architecture with the reconfigurable PE structure.
We also implement TF-MVP design including 4096 multiplier-accumulate operators in a 28nm CMOS technology, achieving higher resource utilization and computational efficiency than the recent sparsity-aware transformer accelerators.
From the fine-grained pruning map, the direction score is developed to analyze the pruning patterns quantitatively for the first time, indicating the major pruning direction and size of each layer.
Then, the mixed-length vector pruning is applied to generate the hardware-friendly pruned-transformer model, which is fully supported by the proposed TF-MVP architecture with the reconfigurable PE structure.
We also implement TF-MVP design including 4096 multiplier-accumulate operators in a 28nm CMOS technology, achieving higher resource utilization and computational efficiency than the recent sparsity-aware transformer accelerators.
Event Type
Research Manuscript
TimeWednesday, July 12th1:55pm - 2:10pm PDT
Location3012, 3rd Floor
Design
AI/ML Architecture Design
