Ray-patch: an efficient querying for light field transformers

Berriel Martins, Tomás ; Civera Sancho, Jsvier

Resumen: In this paper we propose the Ray-Patch querying, a novel model to efficiently query transformers to decode implicit representations into target views. Our Ray-Patch decoding reduces the computational footprint and increases inference speed up to one order of magnitude compared to previous models, without losing global attention, and hence maintaining specific task metrics.
The key idea of our novel querying is to split the target image into a set of patches, then querying the transformer for each patch to extract a set of feature vectors, which are finally decoded into the target image using convolutional layers.
Our experimental results, implementing Ray-Patch in 3 different architectures and evaluating it in 2 different tasks and datasets, demonstrate and quantify the effectiveness of our method, specifically a notable boost in rendering speed for the same task metrics.

Idioma: Inglés

DOI: 10.1109/3DV62453.2024.00124

Año: 2024

Publicado en: Proceedings (International Conference on 3D Vision) (2024), 365-375
ISSN: 2378-3826

Financiación: info:eu-repo/grantAgreement/ES/DGA-FSE/T45-17R
Financiación: info:eu-repo/grantAgreement/ES/MICINN/PID2021-127685NB-I00
Financiación: info:eu-repo/grantAgreement/EUR/MICINN/TED2021-131150B-I00

Tipo y forma: Artículo (PostPrint)