BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:America/Los_Angeles
X-LIC-LOCATION:America/Los_Angeles
BEGIN:DAYLIGHT
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
TZNAME:PDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
TZNAME:PST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20230712T175214Z
LOCATION:3001\, 3rd Floor
DTSTART;TZID=America/Los_Angeles:20230712T140000
DTEND;TZID=America/Los_Angeles:20230712T143000
UID:dac_DAC 2023_sess224_SPSSN123@linklings.com
SUMMARY:Algorithm-Software-Hardware Co-Design for Deep Learning Accelerati
 on
DESCRIPTION:Special Session (Research)\n\nYanzhi Wang (Northeastern Univer
 sity)\n\nThis talk presents the design of efficient and resilient edge Dee
 p Learning computing systems, exploring the promising Transformer architec
 tures, which made attractive resurgence as the Vision Transformer (ViT) an
 d the phenomenal ChatGPT. The unique difficulties in the wide deployment o
 f such large models include (i) the quadratic time and memory complexities
  with respect to the input sequence length, which hinders the scalability 
 on resource-constrained edge devices, and (ii) the ultra-deep architecture
 s of Transformers when presented to the compiler result in highly frequent
  input/output and low resource utilization on edge devices. Based on the n
 ovel Dynamic Vision Transformer (DViT) framework, we overcome the above ch
 allenges with innovations across the algorithm/compiler/hardware layers. A
 t algorithm layer, we propose a latency-driven architecture search for inf
 erence acceleration on top of our DViT framework, aiming for beyond-Mobile
 Net performance at the edge. We develop compiler optimization specific for
  the ultra-deep architectures of Transformers and execution dynamism, achi
 eving another 2X-3X inference speedup. Moreover, we design efficient on-de
 vice learning with two complementary approaches i.e., layer freezing-enabl
 ed dynamic sparse training and low-bit training, achieving 60% less traini
 ng time and 4X training cost reduction, respectively.\n\nTopic: AI\n\nSess
 ion Chair: Mimi Xie (The University of Texas at San Antonio)
END:VEVENT
END:VCALENDAR