Presentation
EagleRec: Edge-Scale Recommendation System Acceleration with Inter-Stage Parallelism Optimization on GPUs
DescriptionRecommendation systems (RecSys) suggest items to users by predicting their preferences based on historical data. The industry traditionally handles large-scale recommendation requests by scaling the number of devices without much concern for a single device's performance. However, there is a trend for RecSys to move from a centralized service to an edge device. In this paper, we focus on the edge-scale RecSys and propose an inter-stage parallelism optimization method. Our approach leverages a utilization-aware multi-stage pipeline to maximize GPU utilization while meeting the service response times. Experiments show that our framework could improve RecSys throughput by 1.9×∼2.6× for different datasets.
Event Type
Research Manuscript
TimeThursday, July 13th11:10am - 11:25am PDT
Location3003, 3rd Floor
AI
AI/ML Application and Infrastructure