110867 20240319080949.0 doi 10.3390/app12031141 sideral 127694 ART-2022-127694 eng Mingote, Victoria Universidad de Zaragoza (orcid)0000-0002-3505-0249 Multimodal Diarization Systems by Training Enrollment Models as Identity Representations 2022 Access copy available to the general public Unrestricted This paper describes a post-evaluation analysis of the system developed by ViVoLAB research group for the IberSPEECH-RTVE 2020 Multimodal Diarization (MD) Challenge. This challenge focuses on the study of multimodal systems for the diarization of audiovisual files and the assignment of an identity to each segment where a person is detected. In this work, we implemented two different subsystems to address this task using the audio and the video from audiovisual files separately. To develop our subsystems, we used the state-of-the-art speaker and face verification embeddings extracted from publicly available deep neural networks (DNN). Different clustering techniques were also employed in combination with the tracking and identity assignment process. Furthermore, we included a novel back-end approach in the face verification subsystem to train an enrollment model for each identity, which we have previously shown to improve the results compared to the average of the enrollment data. Using this approach, we trained a learnable vector to represent each enrollment character. The loss function employed to train this vector was an approximated version of the detection cost function (aDCF) which is inspired by the DCF widely used metric to measure performance in verification tasks. In this paper, we also focused on exploring and analyzing the effect of training this vector with several configurations of this objective loss function. This analysis allows us to assess the impact of the configuration parameters of the loss in the amount and type of errors produced by the system. info:eu-repo/grantAgreement/ES/DGA/T36-20R info:eu-repo/grantAgreement/EC/H2020/101007666/EU/Exchanges for SPEech ReseArch aNd TechnOlogies/ESPERANTO This project has received funding from the European Union’s Horizon 2020 research and innovation program under grant agreement No H2020 101007666-ESPERANTO info:eu-repo/grantAgreement/ES/MCIN/AEI/10.13039/501100011033 info:eu-repo/grantAgreement/ES/MINECO/PRE2018-083312 info:eu-repo/semantics/openAccess by http://creativecommons.org/licenses/by/3.0/es/ 2.7 2022 PHYSICS, APPLIED 78 / 160 = 0.488 2022 Q2 T2 ENGINEERING, MULTIDISCIPLINARY 42 / 90 = 0.467 2022 Q2 T2 CHEMISTRY, MULTIDISCIPLINARY 100 / 178 = 0.562 2022 Q3 T2 MATERIALS SCIENCE, MULTIDISCIPLINARY 208 / 343 = 0.606 2022 Q3 T2 0.492 2022 Fluid Flow and Transfer Processes 2022 Q2 Materials Science (miscellaneous) 2022 Q2 Engineering (miscellaneous) 2022 Q2 Instrumentation 2022 Q2 Process Chemistry and Technology 2022 Q3 Computer Science Applications 2022 Q3 4.5 2022 info:eu-repo/semantics/article info:eu-repo/semantics/publishedVersion Viñals, Ignacio Universidad de Zaragoza (orcid)0000-0001-9137-4013 Gimeno, Pablo Universidad de Zaragoza (orcid)0000-0002-3142-0708 Miguel, Antonio Universidad de Zaragoza (orcid)0000-0001-5803-4316 Ortega, Alfonso Universidad de Zaragoza (orcid)0000-0002-3886-7748 Lleida, Eduardo Universidad de Zaragoza (orcid)0000-0003-1772-0605 5008 800 Universidad de Zaragoza Dpto. Ingeniería Electrón.Com. Área Teoría Señal y Comunicac. 12, 3 (2022), 1141 [15 pp] Appl. sci. Applied Sciences (Switzerland) 2076-3417 572169 http://zaguan.unizar.es/record/110867/files/texto_completo.pdf Versión publicada 2799278 http://zaguan.unizar.es/record/110867/files/texto_completo.jpg?subformat=icon icon Versión publicada oai:zaguan.unizar.es:110867 articulos driver 2024-03-18-12:51:50 ARTICLE