Last summer from August 10 to 14, Professor Sungyoung Kim and our lab member, Kangeun Lee, attended the ACM SIGGRAPH International Conference held in Vancouver, Canada. The conference venue was the Convention Centre, located right next to Vancouver Harbour.
The SIGGRAPH conference offered many sessions to enjoy and learn from. In particular, the Spatial Storytelling and Immersive Pavilion showcased prototypes with innovative ideas and technologies.
At the Immersive Pavilion, Professor and Kangeun had the opportunity to experience a hyper-realistic VR device developed by Reality Labs Research at Meta. Two types of content were provided: A cockpit-view flight simulator and immersive nature experiences. These VR demonstrations showed significant advancement over existing VR devices.
In addition, during the Spatial Storytelling session, researchers from Reality Labs Research at Meta gave a lecture titled “Spatial Storybook: Language Model-Driven Audiobook Spatialization”, with a focus on spatial audio. Throughout the talk, the research team presented their work on LLM-based audiobook spatialization implemented on Meta AI Glasses in collaboration with Ray-Ban. At the end of the lecture, an additional live demonstration using the Meta AI Glasses was presented, offering an immersive storybook experience.
At the Poster session, our lab member Kangeun presented her research as poster entitled
“PAAP: Performer-Aware Automatic Panning System”. This study proposes a fully automated system for audio panning. The proposed approach reduces task complexity of conventional audio panning methods by analyzing input video using computer vision models. Also, a newly developed algorithm is suggested for perspective transformation based on monocular depth feature maps. The system also demonstrates acceptable performance in directly extrapolating panning information from input video captured with a single camera, without relying on multi-view camera setups. This highlights one of the key technical contributions of the proposed research.
The poster presentation of this research provided an excellent opportunity to introduce her work on automatic panning system development at a prestigious international conference. During the session, she engaged in in-depth discussions with other researchers and attendees. Additionally, a lecturer from the Spatial Storytelling session also visited her poster, allowing Kangeun to share her research journey and perspectives. The session served as a valuable networking opportunity.
Prof. Sunyoung Kim and Kangeun enjoyed engaging with the SIGGRAPH community and learning from diverse research perspectives. We anticipate that the connections and ideas gained during the conference will lead to meaningful collaborations and inspire creative future research directions and explorations.