150337 20260114074909.0 Computer Science (miscellaneous) 2024 Q2 Theoretical Computer Science 2024 Q3 0.352 2024 2.4 2024 ARTICLE (orcid)0000-0002-6741-844X Martinez-Cantin, Ruben Universidad de Zaragoza (orcid)0000-0001-5209-2267 Guerrero, Jose J. Universidad de Zaragoza Farinella, Giovanni Maria Furnari, Antonino ART-2024-142469 9282596 http://zaguan.unizar.es/record/150337/files/texto_completo.pdf Postprint info:eu-repo/date/embargoEnd/2025-10-31 1504944 http://zaguan.unizar.es/record/150337/files/texto_completo.jpg?subformat=icon icon Postprint info:eu-repo/date/embargoEnd/2025-10-31 2026-01-12-12:59:37 info:eu-repo/semantics/openAccess All rights reserved http://www.europeana.eu/rights/rr-f/ info:eu-repo/semantics/article info:eu-repo/semantics/acceptedVersion Short-Term object-interaction Anticipation (STA) consists of detecting the location of the next-active objects, the noun and verb categories of the interaction, and the time to contact from the observation of egocentric video. This ability is fundamental for wearable assistants or human-robot interaction to understand the user’s goals, but there is still room for improvement to perform STA in a precise and reliable way. In this work, we improve the performance of STA predictions with two contributions: 1) We propose STAformer, a novel attention-based architecture integrating frame-guided temporal pooling, dual image-video attention, and multiscale feature fusion to support STA predictions from an image-input video pair; 2) We introduce two novel modules to ground STA predictions on human behavior by modeling affordances. First, we integrate an environment affordance model which acts as a persistent memory of interactions that can take place in a given physical scene. Second, we predict interaction hotspots from the observation of hands and object trajectories, increasing confidence in STA predictions localized around the hotspot. Our results show significant relative Overall Top-5 mAP improvements of up to on Ego4D and on a novel set of curated EPIC-Kitchens STA labels. We will release the code, annotations, and pre-extracted affordances on Ego4D and EPIC-Kitchens to encourage future research in this area. Access copy available to the general public Unrestricted 2024 (2024), 167-184 Lect. notes comput. sci. Lecture Notes in Computer Science 0302-9743 5007 520 Universidad de Zaragoza Dpto. Informát.Ingenie.Sistms. Área Ingen.Sistemas y Automát. AFF-ttention! Affordances and Attention Models for Short-Term Object Interaction Anticipation doi 10.1007/978-3-031-73337-6_10 sideral 142469 2024 Mur-Labadia, Lorenzo Universidad de Zaragoza eng oai:zaguan.unizar.es:150337 articulos driver info:eu-repo/grantAgreement/ES/MICINN-AEI/PID2021-125209OB-I00 info:eu-repo/grantAgreement/EUR/MICINN/TED2021-129410B-I00