Technical & Applied Article

Heard-text recall and listening effort under irrelevant speech and pseudo-speech in virtual reality

¹ Institute for Hearing Technology and Acoustics, RWTH Aachen University, Aachen, Germany
² Teaching & Research Area Work and Engineering Psychology, RWTH Aachen University, Aachen, Germany
³ Visual Computing Institute, RWTH Aachen University, Aachen, Germany

^* Corresponding author: This email address is being protected from spambots. You need JavaScript enabled to view it.

Received: 16 September 2025
Accepted: 13 January 2026

Abstract

Introduction: Verbal communication depends on a listener’s ability to accurately comprehend and recall information conveyed in a conversation. The heard-text recall (HTR) paradigm can be used in a dual-task design to assess both memory performance and listening effort. The HTR paradigm uses running speech to simulate a conversation between two talkers. Thereby, it allows for talker visualization in virtual reality (VR), providing co-verbal visual cues like lip-movements, turn-taking cues, and gaze behavior. While the HTR in a dual-task design has been investigated under pink noise, the impact of more realistic irrelevant stimuli, such as speech, that provide temporal fluctuations and meaning compared to noise, remains unexplored.

Methods: In this study (N = 24), the HTR task as primary task was administered in an immersive VR environment under three noise conditions: silence, pseudo-speech, and speech. Participants performed a vibrotactile secondary task to quantify listening effort in a dual-task design.

Results: The results indicate an effect of irrelevant speech on memory and speech comprehension as well as secondary task performance, with a stronger impact of speech relative to pseudo-speech.

Discussion: The study validates the sensitivity of the HTR in a dual-task design to background speech stimuli and highlights the relevance of linguistic interference-by-process for listening effort, speech comprehension, and memory.