SPP 2236 - AUDICTIVE

Evaluating cognitive performance in classroom scenarios using audiovisual virtual reality – ECoClass-VR

In ECoClass-VR, we will investigate the suitability of audiovisual Immersive Virtual Environments (IVE) for a "close-to-real-life" assessment of environmental impacts on cognitive performance of adults and children in classroom-like settings. The existing knowledge about how classroom environments affect cognitive performance predominantly stems from auditory experimental paradigms that have typically used simple acoustic reproductions, with limited consideration of visual processing, consequently leading to poor representations of relevant audiovisual aspects. Hence, to improve the validity of cognitive performance research within classroom-type settings for adults and children, we plan to increase the realism of the experimental procedures, in terms of the cognitive tasks used and the technical settings of the audiovisual representations. For this, two existing cognitive performance assessment paradigms – selective attention and listening comprehension – will be transferred from their audio-only focus to the use of IVE-based complex audiovisual scenes. Another, third paradigm will be investigated, which already incorporates some audiovisual aspects for auditory cognition research. In ECoClass-VR, it will be further adapted to IVE-based classroom-scene assessment, by including more diverse and more complex scenes. The ultimate target group of the research in ECoClass-VR is children. However, due to the additional sensitivity of research with children, the participants during the methodology development phase of ECoClass-VR will be adults only. This will allow choosing the most suitable paradigm out of the three aforementioned paradigms, which will then be used in the last stage in the project: a prototypical study with children. ECoClass-VR primarily addresses the AUDICTIVE research priority (a), auditory cognition, by investigating methodological approaches to assess cognitive performance. Hence, our planned research is highly consistent with the overall AUDICTIVE theme of revisiting auditory-cognitive experimental paradigms that lack in appropriate audiovisual considerations, and scientifically scrutinizing them using state-of-the-art IVEs.The project will include research on the quality of IVEs (AUDICTIVE priority (c)), using established quality and Quality-of-Experience (QoE) measures such as audiovisual quality, presence and simulator sickness as well as changes in cognitive performance for different IVE-variants, complemented by capturing participant behavior during empirical research. Along the project research, indicators of audiovisual cognition and of more traditional aspects of IVE quality assessment are employed to evaluate a variety of IVE instances. This way, ECoClass-VR will contribute to research priority (b), providing scientific and technical insights for IVEs that may enable auditory and audiovisual cognition research for classroom-like settings.