Knowledge Base

Replicating Outdoor Environments Using VR And Ambisonics: A Methodology For Accurate Audio-Visual Recording, Processing And Reproduction

Author (s)

Fotis Georgiou, Claudia Kawai, Beat Schäffer, Reto Pieren

Affiliation

Empa, Swiss Federal Laboratories for Materials Science and Technology, Laboratory for Acoustics/Noise Control, Dübendorf, Switzerland

Publication date

2024

Abstract

This paper introduces a methodology tailored to capture, post-process, and replicate audio-visual data of outdoor environments (urban or natural) for VR experiments carried out within a controlled laboratory environment. The methodology consists of 360∘ video and higher order ambisonic (HOA) field recordings and subsequent calibrated spatial sound reproduction with a spherical loudspeaker array and video played back via a head-mounted display using a game engine and a graphical user interface for a perceptual experimental questionnaire. Attention was given to the equalisation and calibration of the ambisonic microphone and to the design of different ambisonic decoders. A listening experiment was conducted to evaluate four different decoders (one 2D first-order ambisonic decoder and three 3D third-order decoders) by asking participants to rate the relative (perceived) realism of recorded outdoor soundscapes reproduced with these decoders. The results showed that the third-order decoders were ranked as more realistic.

Full paper

https://link.springer.com/article/10.1007/s10055-024-01003-1

Keywords

ambisonics, spatial audio, virtual reality (VR), soundscape reproduction, perceptual evaluation