A Study on the Impacts of Slot Types and Training Data on Joint Natural Language Understanding in a Spanish Medication Management Assistant Scenario

Roca Mainer, S. (Universidad de Zaragoza) ; Rosset, S. ; García Moros, J. (Universidad de Zaragoza) ; Alesanco Iglesias, Á. (Universidad de Zaragoza)

Resumen: This study evaluates the impacts of slot tagging and training data length on joint natural language understanding (NLU) models for medication management scenarios using chatbots in Spanish. In this study, we define the intents (purposes of the sentences) for medication management scenarios and two types of slot tags. For training the model, we generated four datasets, combining long/short sentences with long/short slots, while for testing, we collect the data from real interactions of users with a chatbot. For the comparative analysis, we chose six joint NLU models (SlotRefine, stack-propagation framework, SF-ID network, capsule-NLU, slot-gated modeling, and a joint SLU-LM model) from the literature. The results show that the best performance (with a sentence-level semantic accuracy of 68.6%, an F1-score of 76.4% for slot filling, and an accuracy of 79.3% for intent detection) is achieved using short sentences and short slots. Our results suggest that joint NLU models trained with short slots yield better results than those trained with long slots for the slot filling task. The results also indicate that short slots could be a better choice for the dialog system because of their simplicity. Importantly, the work demonstrates that the performance of the joint NLU models can be improved by selecting the correct slot configuration according to the usage scenario. © 2022 by the authors. Licensee MDPI, Basel, Switzerland.

Idioma: Inglés

DOI: 10.3390/s22062364

Año: 2022

Publicado en: Sensors 22, 6 (2022), 2364 [13 pp]
ISSN: 1424-8220

Factor impacto JCR: 3.9 (2022)

Categ. JCR: CHEMISTRY, ANALYTICAL rank: 26 / 86 = 0.302 (2022) - Q2 - T1
Categ. JCR: INSTRUMENTS & INSTRUMENTATION rank: 19 / 63 = 0.302 (2022) - Q2 - T1
Categ. JCR: ENGINEERING, ELECTRICAL & ELECTRONIC rank: 100 / 274 = 0.365 (2022) - Q2 - T2

Factor impacto CITESCORE: 6.8 - Engineering (Q1) - Chemistry (Q1) - Biochemistry, Genetics and Molecular Biology (Q2) - Physics and Astronomy (Q1)

Factor impacto SCIMAGO: 0.764 - Instrumentation (Q1) - Analytical Chemistry (Q1) - Medicine (miscellaneous) (Q2) - Information Systems (Q2) - Biochemistry (Q2) - Atomic and Molecular Physics, and Optics (Q2) - Electrical and Electronic Engineering (Q2)

Financiación: info:eu-repo/grantAgreement/ES/DGA-FEDER/T31-20R
Financiación: info:eu-repo/grantAgreement/ES/MINECO/BES-2017-082017
Financiación: info:eu-repo/grantAgreement/ES/MINECO-FEDER/TIN2016-76770-R

Tipo y forma: Article (Published version)

Área (Departamento): Área Ingeniería Telemática (Dpto. Ingeniería Electrón.Com.)