Presentation
Latency Processing Unit (LPU) for Acceleration of Hyperscale AI Models
DescriptionTransformers based on hyperscale AI models are innovating the era of AI once again, with two good properties towards artificial general intelligence. First, the performance of transformer models continues to grow with their model sizes and training data. Second, transformers can be pre-trained with unlabeled data through self-supervised learning and can be fine-tuned quickly for each application.
We propose a novel processor named latency processing unit (LPU) to accelerate the end-to-end operations of hyperscale AI models (e.g., OpenAI's GPT). LPU operates on custom instructions and supports entire transformer operations, including multi-head attention, layer normalization, token embedding, and LM head, while maximizing HBM bandwidth with an optimized tiling scheme and dataflow. Multiple instances of LPU can support parallel computation using Intra-layer model parallelism and low-latency peer-to-peer communication. Finally, its multi-FPGA server prototype (HA-X1) achieves 5.58×, 3.99×, and 8.21× improvements in performance, energy efficiency, and cost compared to the comparable GPU appliances, proving its superior efficiency for hyperscale AI models. We are building silicon IPs, SW stack, and utilities for LPU.
We propose a novel processor named latency processing unit (LPU) to accelerate the end-to-end operations of hyperscale AI models (e.g., OpenAI's GPT). LPU operates on custom instructions and supports entire transformer operations, including multi-head attention, layer normalization, token embedding, and LM head, while maximizing HBM bandwidth with an optimized tiling scheme and dataflow. Multiple instances of LPU can support parallel computation using Intra-layer model parallelism and low-latency peer-to-peer communication. Finally, its multi-FPGA server prototype (HA-X1) achieves 5.58×, 3.99×, and 8.21× improvements in performance, energy efficiency, and cost compared to the comparable GPU appliances, proving its superior efficiency for hyperscale AI models. We are building silicon IPs, SW stack, and utilities for LPU.
Event Type
IP
TimeWednesday, July 12th2:45pm - 3:00pm PDT
Location2012, 2nd Floor
IP
RISC-V