Issue |
Acta Acust.
Volume 6, 2022
|
|
---|---|---|
Article Number | 47 | |
Number of page(s) | 19 | |
Section | Audio Signal Processing and Transducers | |
DOI | https://doi.org/10.1051/aacus/2022040 | |
Published online | 12 October 2022 |
Review Article
Spatial audio signal processing for binaural reproduction of recorded acoustic scenes – review and challenges
1
School of Electrical and Computer Engineering, Ben-Gurion University of the Negev, Beer-Sheva 84105, Israel
2
Reality Labs Research, Meta, Redmond, WA 98052, USA
3
International Audio Laboratories Erlangen (a joint institution of the Friedrich Alexander University Erlangen-Nürnberg (FAU) and Fraunhofer IIS), 91058 Erlangen, Germany
4
Applied Psychoacoustics Laboratory (APL), University of Huddersfield, Huddersfield HD1 3DH, United Kingdom
5
Audio and Acoustics Research Group, Microsoft Research, Redmond, WA 98052, USA
6
Audio and Acoustic Signal Processing Group, The Australian National University, Canberra, Australian Capital Territory 2601, Australia
* Corresponding author: br@bgu.ac.il
Received:
25
March
2022
Accepted:
8
September
2022
Spatial audio has been studied for several decades, but has seen much renewed interest recently due to advances in both software and hardware for capture and playback, and the emergence of applications such as virtual reality and augmented reality. This renewed interest has led to the investment of increasing efforts in developing signal processing algorithms for spatial audio, both for capture and for playback. In particular, due to the popularity of headphones and earphones, many spatial audio signal processing methods have dealt with binaural reproduction based on headphone listening. Among these new developments, processing spatial audio signals recorded in real environments using microphone arrays plays an important role. Following this emerging activity, this paper aims to provide a scientific review of recent developments and an outlook for future challenges. This review also proposes a generalized framework for describing spatial audio signal processing for the binaural reproduction of recorded sound. This framework helps to understand the collective progress of the research community, and to identify gaps for future research. It is composed of five main blocks, namely: the acoustic scene, recording, processing, reproduction, and perception and evaluation. First, each block is briefly presented, and then, a comprehensive review of the processing block is provided. This includes topics from simple binaural recording to Ambisonics and perceptually motivated approaches, which focus on careful array configuration and design. Beamforming and parametric-based processing afford more flexible designs and shift the focus to processing and modeling of the sound field. Then, emerging machine- and deep-learning approaches, which take a further step towards flexibility in design, are described. Finally, specific methods for signal transformations such as rotation, translation and enhancement, enabling additional flexibility in reproduction and improvement in the quality of the binaural signal, are presented. The review concludes by highlighting directions for future research.
Key words: Audio signal processing / Spatial audio / Virtual reality / Augmented reality / Array processing
© The Author(s), published by EDP Sciences, 2022
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Current usage metrics show cumulative count of Article Views (full-text article views including HTML views, PDF and ePub downloads, according to the available data) and Abstracts Views on Vision4Press platform.
Data correspond to usage on the plateform after 2015. The current usage metrics is available 48-96 hours after online publication and is updated daily on week days.
Initial download of the metrics may take a while.