Análisis y experimentación de efectos de degradación del rendimiento visual debido a estímulos auditivos en entornos virtuales erosivos.

Jiménez Navarro, Daniel
Serrano Pacheu, Ana (dir.)

Universidad de Zaragoza, EINA, 2022
Departamento de Informática e Ingeniería de Sistemas, Área de Lenguajes y Sistemas Informáticos

Máster Universitario en Robótica, Gráficos y Visión por Computador

Resumen: In recent years, both consumption and people’s interest in Virtual Reality (VR) are increasing dizzily. This innovative technology provides a number of groundbreaking capabilities while has lately become more accessible due to continued hardware development. In VR, the user turns into an active element that can interact in many ways with the virtual environment, which differs from the user passive role settled in traditional media. This interaction occurs naturally once the user is immersed in the virtual world and senses detect what is happening around. Similarly to reality, human perception can be deceived or altered under certain conditions where our senses gather contradictory or too much information. In fact, an audiovisual suppression effect was reported by Malpica et al. (2020) in which it was proved how auditory stimuli can cause loss of visual information. User’s visual performance degrades when spatially incongruent but temporally consistent sounds are listened at once. Our brain perceives both visual and auditory stimuli although some visual data is lost due to neural interactions.
The main goal of this project is to analyze and get a better insight of this audiovisual suppression effect, more concretely, its auditory part. Using the publication previously mentioned as baseline, we create a virtual environment in which both auditory and visual stimuli will be presented to the user. Regarding auditory stimuli, we research how sounds located at the limits of our hearing range can influence the appearance of this effect. Therefore, frequency values associated to hearing limits are obtained for each user and will be used after in the sounds generated throughout the experiment. The participant will encounter not only unimodal stimuli (auditory or visual only) but bimodal (auditory and visual at the same moment) stimuli as well. Bimodal stimuli are dynamically generated in fixed locations keeping temporally consistency, creating the proper conditions under which the audiovisual suppression effect occurs. By keeping stimuli apparition record as well as user performance regarding the moments when any stimuli was perceived, it is possible to check if the user has suffered the suppression effect.
The experiments and the frequency test have been performed by a group of 20 participants. The achieved results manifest that detection and recognition rates of visual stimuli are indeed decreased by almost inaudible sounds. Thereby, the audiovisual suppression effect still occurs with auditory stimuli located at the limits of our hearing range. Surveys fulfilled by the participants demonstrated how the majority of them experimented a great feeling of immersion and presence in the virtual world. Besides, it is not appreciated that the experiment has significant side effects neither drawbacks that disturb participants while making the virtual experience worse. Lastly, it is suggested as future work the analysis of the eye-tracking data recorded during the experiment, in order to study how users behave when barely audible sounds are perceived in VR. With the aim of researching the impact that other factors, such as personal emotions and state of mind, may have on the suppression effect, a couple of appropriate gadgets are proposed as well.

Tipo de Trabajo Académico: Trabajo Fin de Master