Off-Line Evaluation of Mobile-Centric Indoor Positioning Systems: The Experiences from the 2017 IPIN Competition

The development of indoor positioning solutions using smartphones is a growing activity with an enormous potential for everyday life and professional applications. The research activities on this topic concentrate on the development of new positioning solutions that are tested in specific environmen...

Full description

Bibliographic Details
Authors: Torres-Sospedra, Joaquín, Jiménez Ruiz, Antonio R., Moreira, Adriano, Lungenstrass, Tomás, Lu, Wei-Chung, Knauth, Stefan, Mendoza-Silva, Germán Martín, Seco Granja, Fernando, Pérez-Navarro, Antoni, João Nicolau, Maria, Costa, António, Meneses, Filipe, Farina, Joaquín, Morales, Juan Pablo, Lu, Wen-Chen, Cheng, Ho-Ti, Yang, Shi-Shen, Fang, Shih-Hau, Chien, Ying-Ren, Tsao, Yu
Format: article
Status:Published version
Publication Date:2018
Country:España
Institution:Consejo Superior de Investigaciones Científicas (CSIC)
Repository:DIGITAL.CSIC. Repositorio Institucional del CSIC
OAI Identifier:oai:digital.csic.es:10261/170687
Online Access:http://hdl.handle.net/10261/170687
Access Level:Open access
Keyword:indoor positioning and navigation
Wi-Fi fingerprinting
sensor fusion
Competitions
Benchmarking
id ES_cc9a2d35caf5fe2bf58e9353324a6d56
oai_identifier_str oai:digital.csic.es:10261/170687
network_acronym_str ES
network_name_str España
repository_id_str
spelling Off-Line Evaluation of Mobile-Centric Indoor Positioning Systems: The Experiences from the 2017 IPIN CompetitionTorres-Sospedra, JoaquínJiménez Ruiz, Antonio R.Moreira, AdrianoLungenstrass, TomásLu, Wei-ChungKnauth, StefanMendoza-Silva, Germán MartínSeco Granja, FernandoPérez-Navarro, AntoniJoão Nicolau, MariaCosta, AntónioMeneses, FilipeFarina, JoaquínMorales, Juan PabloLu, Wen-ChenCheng, Ho-TiYang, Shi-ShenFang, Shih-HauChien, Ying-RenTsao, Yuindoor positioning and navigationWi-Fi fingerprintingsensor fusionCompetitionsBenchmarkingThe development of indoor positioning solutions using smartphones is a growing activity with an enormous potential for everyday life and professional applications. The research activities on this topic concentrate on the development of new positioning solutions that are tested in specific environments under their own evaluation metrics. To explore the real positioning quality of smartphone-based solutions and their capabilities for seamlessly adapting to different scenarios, it is needed to find fair evaluation frameworks. The design of competitions using extensive pre-recorded datasets is a valid way to generate open data for comparing the different solutions created by research teams. In this paper, we discuss the details of the 2017 IPIN indoor localization competition, the different datasets created, the teams participating in the event, and the results they obtained. We compare these results with other competition-based approaches (Microsoft and Perf-loc) and on-line evaluation web sites. The lessons learned by organising these competitions and the benefits for the community are addressed along the paper. Our analysis paves the way for future developments on the standardization of evaluations and for creating a widely-adopted benchmark strategy for researchers and companies in the field.We would like to thank Topcon Corporation for sponsoring the competition track with an award for the winning team. We are also grateful to Francesco Potortì, Sangjoon Park, Hideo Makino, Nobuo Kawaguchi, Takeshi Kurata and Jesús Ureña for their invaluable help in organizing and promoting the IPIN competition and conference. Many thanks to Raúl Montoliu, Emilio Sansano, Marina Granel and Luis Alisandra for collecting the databases in the UJITI building. Parts of this work were carried out with the financial support received from projects and grants: REPNIN network (TEC2015-71426-REDT), LORIS (TIN2012-38080-C04-04), TARSIUS (TIN2015-71564-C4-2-R (MINECO/FEDER)), SmartLoc (CSIC-PIE Ref.201450E011), “Metodologías avanzadas para el diseño, desarrollo, evaluación e integración de algoritmos de localización en interiores” (TIN2015-70202-P), GEO-C (Project ID: 642332, H2020-MSCA-ITN-2014—Marie Sklodowska-Curie Action: Innovative Training Networks), and financial support from the Ministry of Science and Technology, Taiwan (106-3114-E-007-005 and 105-2221-E-155-013-MY3). The HFTS team has been supported in the frame of the German Federal Ministry of Education and Research programme “FHprofUnt2013” under contract 03FH035PB3 (Project SPIRIT). The UMinho team has been supported by COMPETE: POCI-01-0145-FEDER-007043 and FCT—Fundação para a Ciência e Tecnologia within the Project Scope: UID/CEC/00319/2013. G. M. Mendoza-Silva gratefully acknowledges funding from grant PREDOC/2016/55 by Universitat Jaume I.Peer ReviewedMolecular Diversity Preservation InternationalMinisterio de Economía, Industria y Competitividad (España)Consejo Superior de Investigaciones Científicas [https://ror.org/02gfc7t72]2018201820182018info:eu-repo/semantics/articlehttp://purl.org/coar/resource_type/c_6501Publisher's versioninfo:eu-repo/semantics/publishedVersionhttp://hdl.handle.net/10261/170687reponame:DIGITAL.CSIC. Repositorio Institucional del CSICinstname:Consejo Superior de Investigaciones Científicas (CSIC)Inglés#PLACEHOLDER_PARENT_METADATA_VALUE#MINECO/TIN2015/71564-C4-2-R (Tarsius)Síinfo:eu-repo/semantics/openAccessoai:digital.csic.es:10261/1706872026-05-22T06:33:51Z
dc.title.none.fl_str_mv Off-Line Evaluation of Mobile-Centric Indoor Positioning Systems: The Experiences from the 2017 IPIN Competition
title Off-Line Evaluation of Mobile-Centric Indoor Positioning Systems: The Experiences from the 2017 IPIN Competition
spellingShingle Off-Line Evaluation of Mobile-Centric Indoor Positioning Systems: The Experiences from the 2017 IPIN Competition
Torres-Sospedra, Joaquín
indoor positioning and navigation
Wi-Fi fingerprinting
sensor fusion
Competitions
Benchmarking
title_short Off-Line Evaluation of Mobile-Centric Indoor Positioning Systems: The Experiences from the 2017 IPIN Competition
title_full Off-Line Evaluation of Mobile-Centric Indoor Positioning Systems: The Experiences from the 2017 IPIN Competition
title_fullStr Off-Line Evaluation of Mobile-Centric Indoor Positioning Systems: The Experiences from the 2017 IPIN Competition
title_full_unstemmed Off-Line Evaluation of Mobile-Centric Indoor Positioning Systems: The Experiences from the 2017 IPIN Competition
title_sort Off-Line Evaluation of Mobile-Centric Indoor Positioning Systems: The Experiences from the 2017 IPIN Competition
dc.creator.none.fl_str_mv Torres-Sospedra, Joaquín
Jiménez Ruiz, Antonio R.
Moreira, Adriano
Lungenstrass, Tomás
Lu, Wei-Chung
Knauth, Stefan
Mendoza-Silva, Germán Martín
Seco Granja, Fernando
Pérez-Navarro, Antoni
João Nicolau, Maria
Costa, António
Meneses, Filipe
Farina, Joaquín
Morales, Juan Pablo
Lu, Wen-Chen
Cheng, Ho-Ti
Yang, Shi-Shen
Fang, Shih-Hau
Chien, Ying-Ren
Tsao, Yu
author Torres-Sospedra, Joaquín
author_facet Torres-Sospedra, Joaquín
Jiménez Ruiz, Antonio R.
Moreira, Adriano
Lungenstrass, Tomás
Lu, Wei-Chung
Knauth, Stefan
Mendoza-Silva, Germán Martín
Seco Granja, Fernando
Pérez-Navarro, Antoni
João Nicolau, Maria
Costa, António
Meneses, Filipe
Farina, Joaquín
Morales, Juan Pablo
Lu, Wen-Chen
Cheng, Ho-Ti
Yang, Shi-Shen
Fang, Shih-Hau
Chien, Ying-Ren
Tsao, Yu
author_role author
author2 Jiménez Ruiz, Antonio R.
Moreira, Adriano
Lungenstrass, Tomás
Lu, Wei-Chung
Knauth, Stefan
Mendoza-Silva, Germán Martín
Seco Granja, Fernando
Pérez-Navarro, Antoni
João Nicolau, Maria
Costa, António
Meneses, Filipe
Farina, Joaquín
Morales, Juan Pablo
Lu, Wen-Chen
Cheng, Ho-Ti
Yang, Shi-Shen
Fang, Shih-Hau
Chien, Ying-Ren
Tsao, Yu
author2_role author
author
author
author
author
author
author
author
author
author
author
author
author
author
author
author
author
author
author
dc.contributor.none.fl_str_mv Ministerio de Economía, Industria y Competitividad (España)
Consejo Superior de Investigaciones Científicas [https://ror.org/02gfc7t72]
dc.subject.none.fl_str_mv indoor positioning and navigation
Wi-Fi fingerprinting
sensor fusion
Competitions
Benchmarking
topic indoor positioning and navigation
Wi-Fi fingerprinting
sensor fusion
Competitions
Benchmarking
description The development of indoor positioning solutions using smartphones is a growing activity with an enormous potential for everyday life and professional applications. The research activities on this topic concentrate on the development of new positioning solutions that are tested in specific environments under their own evaluation metrics. To explore the real positioning quality of smartphone-based solutions and their capabilities for seamlessly adapting to different scenarios, it is needed to find fair evaluation frameworks. The design of competitions using extensive pre-recorded datasets is a valid way to generate open data for comparing the different solutions created by research teams. In this paper, we discuss the details of the 2017 IPIN indoor localization competition, the different datasets created, the teams participating in the event, and the results they obtained. We compare these results with other competition-based approaches (Microsoft and Perf-loc) and on-line evaluation web sites. The lessons learned by organising these competitions and the benefits for the community are addressed along the paper. Our analysis paves the way for future developments on the standardization of evaluations and for creating a widely-adopted benchmark strategy for researchers and companies in the field.
publishDate 2018
dc.date.none.fl_str_mv 2018
2018
2018
2018
dc.type.none.fl_str_mv info:eu-repo/semantics/article
http://purl.org/coar/resource_type/c_6501
Publisher's version
info:eu-repo/semantics/publishedVersion
format article
status_str publishedVersion
dc.identifier.none.fl_str_mv http://hdl.handle.net/10261/170687
url http://hdl.handle.net/10261/170687
dc.language.none.fl_str_mv Inglés
language_invalid_str_mv Inglés
dc.relation.none.fl_str_mv #PLACEHOLDER_PARENT_METADATA_VALUE#
MINECO/TIN2015/71564-C4-2-R (Tarsius)
Sí
dc.rights.none.fl_str_mv info:eu-repo/semantics/openAccess
eu_rights_str_mv openAccess
dc.publisher.none.fl_str_mv Molecular Diversity Preservation International
publisher.none.fl_str_mv Molecular Diversity Preservation International
dc.source.none.fl_str_mv reponame:DIGITAL.CSIC. Repositorio Institucional del CSIC
instname:Consejo Superior de Investigaciones Científicas (CSIC)
instname_str Consejo Superior de Investigaciones Científicas (CSIC)
reponame_str DIGITAL.CSIC. Repositorio Institucional del CSIC
collection DIGITAL.CSIC. Repositorio Institucional del CSIC
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869419734912466944
score 15.228081

Similar Items