The Importance of AI Efficiency in AR/VR

AI City network server technology

Department of Electrical and Computer Engineering

Location: Gateway South 021

Speaker: Sai Qian Zhang, Assistant Professor, Electrical Engineering and Computer Science, New York University

ABSTRACT

The algorithmic power of DNNs often comes at a high cost in terms of latency and energy consumption, posing significant challenges, especially in resource-constrained environments like AR/VR. AR/VR applications demand real-time processing to deliver immersive and interactive experiences, which not only require high-performance systems but also solutions that prioritize low latency and energy efficiency. To overcome these challenges, it is crucial to design a full-stack solution that co-optimizes DNN algorithms with the underlying hardware to unlock the full potential of AR/VR, enabling more responsive and energy-efficient systems capable of meeting the performance demands of next-generation immersive technologies. In this talk, I will first describe gaze-tracked foveated rendering, where gaze tracking is typically predicted using DNNs, and emphasize the necessity of co-optimizing the gaze-tracking DNN with the underlying hardware platform to accelerate the rendering process. Finally, I will provide an overview of some of the general work being conducted in my group on the efficient design of LLMs, highlighting how we are optimizing architectures and hardware to improve their efficiency and scalability for a range of applications.

BIOGRAPHY

Portrait of Sai Qian Zhang

Sai Qian Zhang is an Assistant Professor of Electrical Engineering and Computer Science at New York University. Prior to joining NYU, he spent two years at Reality Labs at Meta. Sai earned his Ph.D. from Harvard University in 2021 and holds both M.A.Sc and B.A.Sc degrees from the University of Toronto. His research interests include efficient machine learning algorithms and systems and AR/VR computing.