Estudios
I+D+I
Institución
Internacional
Vida Universitaria
Universidad de Zaragoza Repository
Search
Submit
Personalize
Your alerts
Your baskets
Your searches
Help
EN
/
ES
Home
>
Articles
> A multimodal dataset for object model learning from natural human-robot interaction
Usage statistics
Plots
A multimodal dataset for object model learning from natural human-robot interaction
Azagra, P.
(Universidad de Zaragoza)
;
Golemo, F.
;
Mollard, Y.
;
Lopes, M.
;
Civera, J.
(Universidad de Zaragoza)
;
Murillo, A.C.
(Universidad de Zaragoza)
Resumen:
Learning object models in the wild from natural human interactions is an essential ability for robots to perform general tasks. In this paper we present a robocentric multimodal dataset addressing this key challenge. Our dataset focuses on interactions where the user teaches new objects to the robot in various ways. It contains synchronized recordings of visual (3 cameras) and audio data which provide a challenging evaluation framework for different tasks. Additionally, we present an end-to-end system that learns object models using object patches extracted from the recorded natural interactions. Our proposed pipeline follows these steps: (a) recognizing the interaction type, (b) detecting the object that the interaction is focusing on, and (c) learning the models from the extracted data. Our main contribution lies in the steps towards identifying the target object patches of the images. We demonstrate the advantages of combining language and visual features for the interaction recognition and use multiple views to improve the object modelling. Our experimental results show that our dataset is challenging due to occlusions and domain change with respect to typical object learning frameworks. The performance of common out-of-the-box classifiers trained on our data is low. We demonstrate that our algorithm outperforms such baselines.
Idioma:
Inglés
DOI:
10.1109/IROS.2017.8206514
Año:
2017
Publicado en:
Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems
2017, 9-17418366 (2017), 6134-6141
ISSN:
2153-0858
Financiación:
info:eu-repo/grantAgreement/ES/DGA/T04
Financiación:
info:eu-repo/grantAgreement/EUR/FP7/PCIN-2015-122
Financiación:
info:eu-repo/grantAgreement/EC/FP7/248663/EU/European Coordinated Research on Long-term Challenges in Information and Communication Sciences and Technologies/CHIST- ERA
Financiación:
info:eu-repo/grantAgreement/EC/FP7/610878/EU/Semi-Autonomous 3rd Hand/3rd HAND
Financiación:
info:eu-repo/grantAgreement/ES/MINECO/DPI2015-65962-R
Financiación:
info:eu-repo/grantAgreement/ES/MINECO/DPI2015-67275
Financiación:
info:eu-repo/grantAgreement/ES/MINECO-FEDER/DPI2015-69376-R
Financiación:
info:eu-repo/grantAgreement/ES/UZ/JIUZ-2015-TEC-03
Tipo y forma:
Article (PostPrint)
Área (Departamento):
Área Ingen.Sistemas y Automát.
(
Dpto. Informát.Ingenie.Sistms.
)
All rights reserved by journal editor
Exportado de SIDERAL (2021-10-08-11:37:30)
Permalink:
Copy
Visitas y descargas
Este artículo se encuentra en las siguientes colecciones:
Articles
Back to search
Record created 2018-12-14, last modified 2021-10-08
Postprint:
PDF
Rate this document:
Rate this document:
1
2
3
4
5
(Not yet reviewed)
Add to personal basket
Export as
BibTeX
,
MARC
,
MARCXML
,
DC
,
EndNote
,
NLM
,
RefWorks