tSPM-Net: A probabilistic spatio-temporal approach for scanpath prediction

Martín, Daniel (Universidad de Zaragoza) ; Gutiérrez, Diego (Universidad de Zaragoza) ; Masia, Belén (Universidad de Zaragoza)

Resumen: Predicting the path followed by the viewer’s eyes when observing an image (a scanpath) is a challenging problem, particularly due to the inter- and intra-observer variability and the spatio-temporal dependencies of the visual attention process. Most existing approaches have focused on progressively optimizing the prediction of a gaze point given the previous ones. In this work we propose instead a probabilistic approach, which we call tSPM-Net. We build our method to account for observers’ variability by resorting to Bayesian deep learning and a probabilistic approach. Besides, we optimize our model to jointly consider both spatial and temporal dimensions of scanpaths using a novel spatio-temporal loss function based on a combination of Kullback–Leibler divergence and dynamic time warping. Our tSPM-Net yields results that outperform those of current state-of-the-art approaches, and are closer to the human baseline, suggesting that our model is able to generate scanpaths whose behavior closely resembles those of the real ones.

Idioma: Inglés

DOI: 10.1016/j.cag.2024.103983

Año: 2024

Publicado en: COMPUTERS & GRAPHICS-UK 122 (2024), 103983 [9 pp.]
ISSN: 0097-8493

Factor impacto JCR: 2.8 (2024)

Categ. JCR: COMPUTER SCIENCE, SOFTWARE ENGINEERING rank: 53 / 129 = 0.411 (2024) - Q2 - T2

Factor impacto CITESCORE: 6.1 - Computer Graphics and Computer-Aided Design (Q1) - Computer Vision and Pattern Recognition (Q1) - Engineering (all) (Q1) - Signal Processing (Q1) - Human-Computer Interaction (Q2) - Software (Q2)

Factor impacto SCIMAGO: 0.569 - Computer Graphics and Computer-Aided Design (Q2) - Computer Vision and Pattern Recognition (Q2) - Software (Q2) - Engineering (miscellaneous) (Q2) - Signal Processing (Q2) - Human-Computer Interaction (Q3)

Financiación: info:eu-repo/grantAgreement/ES/AEI/PID2022-141539NB-I00
Financiación: info:eu-repo/grantAgreement/ES/DGA/T34-20R
Financiación: info:eu-repo/grantAgreement/EC/H2020/956585/EU/Predictive Rendering In Manufacture and Engineering/PRIME

Tipo y forma: Artículo (Versión definitiva)

Área (Departamento): Área Lenguajes y Sistemas Inf. (Dpto. Informát.Ingenie.Sistms.)