A comparative analysis of early and late fusion for the multimodal two-class problem

[EN] In this article we carry out a comparison between early (feature) and late (score) multimodal fusion, for the two-class problem. The comparison is made first from a general perspective, and then from a specific mathematical analysis. Thus, we deduce the error probability expressions for the unc...

ver descrição completa

Detalhes bibliográficos
Autores: Pereira-González, Luis Manuel, Salazar Afanador, Addisson|||0000-0001-5849-5104, Vergara Domínguez, Luís|||0000-0001-6803-4774
Formato: artículo
Fecha de publicación:2023
País:España
Recursos:Universitat Politècnica de València (UPV)
Repositorio:RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Idioma:inglés
OAI Identifier:oai:riunet.upv.es:10251/205379
Acesso em linha:https://riunet.upv.es/handle/10251/205379
Access Level:acceso abierto
Palavra-chave:Multimodal two-class classification
Early fusion
Late fusion
Probability of error
Training set size
TEORÍA DE LA SEÑAL Y COMUNICACIONES
id ES_fc3841d33caee99b8d5914ac3629b0e9
oai_identifier_str oai:riunet.upv.es:10251/205379
network_acronym_str ES
network_name_str España
repository_id_str
spelling A comparative analysis of early and late fusion for the multimodal two-class problemPereira-González, Luis ManuelSalazar Afanador, Addisson|||0000-0001-5849-5104Vergara Domínguez, Luís|||0000-0001-6803-4774Multimodal two-class classificationEarly fusionLate fusionProbability of errorTraining set sizeTEORÍA DE LA SEÑAL Y COMUNICACIONES[EN] In this article we carry out a comparison between early (feature) and late (score) multimodal fusion, for the two-class problem. The comparison is made first from a general perspective, and then from a specific mathematical analysis. Thus, we deduce the error probability expressions for the uncorrelated and correlated multivariate Gaussian distribution, assuming perfect model knowledge (Bayes error rates). We also deduce the corresponding expressions when the model is to be learned from a finite training set, demonstrating its convergence to the Bayes error rates as the training set size goes to infinite. These expressions also demonstrates that early fusion is the best option with model knowledge, and that both early and late fusion degrade due to a finite training set. This degradation is showed to be greater for early fusion due to the dimensionality increase of the feature space, so, eventually, late fusion could be a better option in a practical setting. The mathematical analysis also suggests the convenience of using a, so called, convergence factor, to quantify if a training set size is appropriate for the error probability to be close enough to the Bayes error rate. Different simulated experiments have been made to verify the validity of the mathematical analysis, as well as its possible extension to non-Gaussian models.This work was supported in part by MCIN/AEI/10.13039/501100011033 under Grant PRE2018-085092, and in part by Universitat Politecnica de Valencia.Institute of Electrical and Electronics EngineersEscuela Técnica Superior de Ingeniería de TelecomunicaciónDepartamento de ComunicacionesInstituto Universitario de Telecomunicación y Aplicaciones MultimediaAGENCIA ESTATAL DE INVESTIGACIONUniversitat Politècnica de ValènciaRepositorio Institucional de la Universitat Politècnica de València Riunet20232023-01-01journal articlehttp://purl.org/coar/resource_type/c_6501VoRhttp://purl.org/coar/version/c_970fb48d4fbd8a85info:eu-repo/semantics/articleapplication/pdfhttps://riunet.upv.es/handle/10251/205379reponame:RiuNet. Repositorio Institucional de la Universitat Politécnica de Valénciainstname:Universitat Politècnica de València (UPV)InglésengAgencia Estatal de Investigación http://dx.doi.org/10.13039/501100011033 Plan Estatal de Investigación Científica y Técnica y de Innovación 2013-2016 TEC2017-84743-P METODOS INFORMADOS PARA LA SINTESIS DE SEÑALESAgencia Estatal de Investigación http://dx.doi.org/10.13039/501100011033 PRE2018-085092 AYUDA PARA CONTRATOS PREDOCTORALES PARA LA FORMACION DE DOCTORES-PEREIRA GONZALEZ, LUIS. PROYECTO: METODOS INFORMADOS PARA LA SINTESIS DE SEÑALESopen accesshttp://purl.org/coar/access_right/c_abf2Reconocimiento - No comercial - Sin obra derivada (by-nc-nd) http://creativecommons.org/licenses/by-nc-nd/4.0/info:eu-repo/semantics/openAccessoai:riunet.upv.es:10251/2053792026-06-13T07:49:27Z
dc.title.none.fl_str_mv A comparative analysis of early and late fusion for the multimodal two-class problem
title A comparative analysis of early and late fusion for the multimodal two-class problem
spellingShingle A comparative analysis of early and late fusion for the multimodal two-class problem
Pereira-González, Luis Manuel
Multimodal two-class classification
Early fusion
Late fusion
Probability of error
Training set size
TEORÍA DE LA SEÑAL Y COMUNICACIONES
title_short A comparative analysis of early and late fusion for the multimodal two-class problem
title_full A comparative analysis of early and late fusion for the multimodal two-class problem
title_fullStr A comparative analysis of early and late fusion for the multimodal two-class problem
title_full_unstemmed A comparative analysis of early and late fusion for the multimodal two-class problem
title_sort A comparative analysis of early and late fusion for the multimodal two-class problem
dc.creator.none.fl_str_mv Pereira-González, Luis Manuel
Salazar Afanador, Addisson|||0000-0001-5849-5104
Vergara Domínguez, Luís|||0000-0001-6803-4774
author Pereira-González, Luis Manuel
author_facet Pereira-González, Luis Manuel
Salazar Afanador, Addisson|||0000-0001-5849-5104
Vergara Domínguez, Luís|||0000-0001-6803-4774
author_role author
author2 Salazar Afanador, Addisson|||0000-0001-5849-5104
Vergara Domínguez, Luís|||0000-0001-6803-4774
author2_role author
author
dc.contributor.none.fl_str_mv Escuela Técnica Superior de Ingeniería de Telecomunicación
Departamento de Comunicaciones
Instituto Universitario de Telecomunicación y Aplicaciones Multimedia
AGENCIA ESTATAL DE INVESTIGACION
Universitat Politècnica de València
Repositorio Institucional de la Universitat Politècnica de València Riunet
dc.subject.none.fl_str_mv Multimodal two-class classification
Early fusion
Late fusion
Probability of error
Training set size
TEORÍA DE LA SEÑAL Y COMUNICACIONES
topic Multimodal two-class classification
Early fusion
Late fusion
Probability of error
Training set size
TEORÍA DE LA SEÑAL Y COMUNICACIONES
description [EN] In this article we carry out a comparison between early (feature) and late (score) multimodal fusion, for the two-class problem. The comparison is made first from a general perspective, and then from a specific mathematical analysis. Thus, we deduce the error probability expressions for the uncorrelated and correlated multivariate Gaussian distribution, assuming perfect model knowledge (Bayes error rates). We also deduce the corresponding expressions when the model is to be learned from a finite training set, demonstrating its convergence to the Bayes error rates as the training set size goes to infinite. These expressions also demonstrates that early fusion is the best option with model knowledge, and that both early and late fusion degrade due to a finite training set. This degradation is showed to be greater for early fusion due to the dimensionality increase of the feature space, so, eventually, late fusion could be a better option in a practical setting. The mathematical analysis also suggests the convenience of using a, so called, convergence factor, to quantify if a training set size is appropriate for the error probability to be close enough to the Bayes error rate. Different simulated experiments have been made to verify the validity of the mathematical analysis, as well as its possible extension to non-Gaussian models.
publishDate 2023
dc.date.none.fl_str_mv 2023
2023-01-01
dc.type.none.fl_str_mv journal article
http://purl.org/coar/resource_type/c_6501
VoR
http://purl.org/coar/version/c_970fb48d4fbd8a85
dc.type.openaire.fl_str_mv info:eu-repo/semantics/article
format article
dc.identifier.none.fl_str_mv https://riunet.upv.es/handle/10251/205379
url https://riunet.upv.es/handle/10251/205379
dc.language.none.fl_str_mv Inglés
eng
language_invalid_str_mv Inglés
language eng
dc.relation.none.fl_str_mv Agencia Estatal de Investigación http://dx.doi.org/10.13039/501100011033 Plan Estatal de Investigación Científica y Técnica y de Innovación 2013-2016 TEC2017-84743-P METODOS INFORMADOS PARA LA SINTESIS DE SEÑALES
Agencia Estatal de Investigación http://dx.doi.org/10.13039/501100011033 PRE2018-085092 AYUDA PARA CONTRATOS PREDOCTORALES PARA LA FORMACION DE DOCTORES-PEREIRA GONZALEZ, LUIS. PROYECTO: METODOS INFORMADOS PARA LA SINTESIS DE SEÑALES
dc.rights.none.fl_str_mv open access
http://purl.org/coar/access_right/c_abf2
Reconocimiento - No comercial - Sin obra derivada (by-nc-nd)
http://creativecommons.org/licenses/by-nc-nd/4.0/
dc.rights.openaire.fl_str_mv info:eu-repo/semantics/openAccess
rights_invalid_str_mv open access
http://purl.org/coar/access_right/c_abf2
Reconocimiento - No comercial - Sin obra derivada (by-nc-nd)
http://creativecommons.org/licenses/by-nc-nd/4.0/
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv Institute of Electrical and Electronics Engineers
publisher.none.fl_str_mv Institute of Electrical and Electronics Engineers
dc.source.none.fl_str_mv reponame:RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
instname:Universitat Politècnica de València (UPV)
instname_str Universitat Politècnica de València (UPV)
reponame_str RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
collection RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869425401918390272
score 15,812455

Registos relacionados