Crossmodal perception in virtual reality

Malpica, S. (Universidad de Zaragoza) ; Serrano, A. (Universidad de Zaragoza) ; Allue, M. ; González Bedia, M. (Universidad de Zaragoza) ; Masia, B. (Universidad de Zaragoza)

Resumen: With the proliferation of low-cost, consumer level, head-mounted displays (HMDs) we are witnessing a reappearance of virtual reality. However, there are still important stumbling blocks that hinder the achievable visual quality of the results. Knowledge of human perception in virtual environments can help overcome these limitations. In this work, within the much-studied area of perception in virtual environments, we look into the less explored area of crossmodal perception, that is, the interaction of different senses when perceiving the environment. In particular, we look at the influence of sound on visual perception in a virtual reality scenario. First, we assert the existence of a crossmodal visuo-auditory effect in a VR scenario through two experiments, and find that, similar to what has been reported in conventional displays, our visual perception is affected by auditory stimuli in a VR setup. The crossmodal effect in VR is, however, lower than that present in a conventional display counterpart. Having asserted the effect, a third experiment looks at visuo-auditory crossmodality in the context of material appearance perception. We test different rendering qualities, together with the presence of sound, for a series of materials. The goal of the third experiment is twofold: testing whether known interactions in traditional displays hold in VR, and finding insights that can have practical applications in VR content generation (e.g., by reducing rendering costs).

Idioma: Inglés

DOI: 10.1007/s11042-019-7331-z

Año: 2020

Publicado en: Multimedia Tools and Applications 79 (2020), 3311-3331
ISSN: 1380-7501

Factor impacto JCR: 2.757 (2020)

Categ. JCR: COMPUTER SCIENCE, INFORMATION SYSTEMS rank: 81 / 161 = 0.503 (2020) - Q3 - T2
Categ. JCR: ENGINEERING, ELECTRICAL & ELECTRONIC rank: 120 / 273 = 0.44 (2020) - Q2 - T2
Categ. JCR: COMPUTER SCIENCE, THEORY & METHODS rank: 29 / 110 = 0.264 (2020) - Q2 - T1
Categ. JCR: COMPUTER SCIENCE, SOFTWARE ENGINEERING rank: 30 / 108 = 0.278 (2020) - Q2 - T1

Factor impacto SCIMAGO: 0.443 - Computer Networks and Communications (Q1) - Software (Q1) - Media Technology (Q1) - Hardware and Architecture (Q1)

Tipo y forma: Artículo (PostPrint)

Área (Departamento): Área Lenguajes y Sistemas Inf. (Dpto. Informát.Ingenie.Sistms.)