Artificial intelligence for virtual reality: a review

Abstract

As hardware and information technology continually advance, virtual reality (VR) has permeated numerous sectors, with applications becoming increasingly sophisticated. The evolution of VR systems has expanded from the seminal 3I characteristics—immersion, interaction, and imagination—to encompass 6I, incorporating intelligentization, interconnection, and iteration. The intelligentization of VR technology, an inevitable progression, has garnered heightened interest, particularly fueled by the emergence of artificial intelligence (AI) models and techniques like neural radiance fields, 3D Gaussian Splatting, neural rendering, generative adversarial networks, diffusion models, and large language models, which significantly propel the development of VR’s core and pivotal technologies. This survey offers a comprehensive assessment of these pivotal VR technologies, harnessing the latest AI advancements, aiming to provide fresh perspectives and assist new researchers in staying abreast of groundbreaking work. We commence by detailing the acquisition process of reviewed papers, outlining our taxonomy grounded in VR’s core elements and pivotal technological trajectories, and statistically analyzing the works within. Subsequently, we delve into the application of AI models, methodologies, and techniques across six research avenues: advanced AI-generated content representation, content rendering, content generation, physical simulation, virtual characters, and interaction, discussing their achievements. Concludingly, we summarize our findings, highlight existing challenges, and suggest potential avenues for future research.

Chi Wang 王驰

Distinguished Research Fellow
(ZJU tenure-track positions)

My research interests include AIGC, AIGC, Computer Graphics, 3D Vision, Digital Twin.