Part-of-speech and prosody-based approaches for robot speech and gesture synchronization
Humanoid robots are already among us and they are beginning to assume more social and personal roles, like guiding and assisting people. Thus, they should interact in a human-friendly manner, using not only verbal cues but also synchronized non-verbal and para-verbal cues. However, available robots...
| Autores: | , , |
|---|---|
| Tipo de documento: | artigo |
| Estado: | Versión aceptada para publicación |
| Data de publicação: | 2019 |
| País: | España |
| Recursos: | Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya) |
| Repositório: | Recercat. Dipósit de la Recerca de Catalunya |
| OAI Identifier: | oai:recercat.cat:10230/42897 |
| Acesso em linha: | http://hdl.handle.net/10230/42897 http://dx.doi.org/10.1007/s10846-019-01100-3 |
| Access Level: | Acceso aberto |
| Palavra-chave: | Human-computer interaction Multimodal interaction Humanoid robots Prosody Speech Gesture modelling Arm gesture synthesis Speech and gesture synchronization Text-to-gesture |
| id |
ES_0f99384c1e3fa1dcf5a6ef4fe40ace7b |
|---|---|
| oai_identifier_str |
oai:recercat.cat:10230/42897 |
| network_acronym_str |
ES |
| network_name_str |
España |
| repository_id_str |
|
| spelling |
Part-of-speech and prosody-based approaches for robot speech and gesture synchronizationPérez Mayos, LauraFarrús, MireiaAdell, JordiHuman-computer interactionMultimodal interactionHumanoid robotsProsodySpeechGesture modellingArm gesture synthesisSpeech and gesture synchronizationText-to-gestureHumanoid robots are already among us and they are beginning to assume more social and personal roles, like guiding and assisting people. Thus, they should interact in a human-friendly manner, using not only verbal cues but also synchronized non-verbal and para-verbal cues. However, available robots are not able to communicate in this multimodal way, being just able to perform predefined gesture sequences, hand- crafted to accompany specific utterances. In the current paper, we propose a model based on three different approaches to extend humanoid robots communication behaviour with upper body gestures synchronized with the speech for novel utterances, exploiting part-of-speech grammatical information, prosody cues, and a combination of both. User studies confirm that our methods are able to produce natural, appropriate and good timed gesture sequences synchronized with speech, using both beat and emblematic gestures.The second author has been funded by the Agencia Estatal de Investigación (AEI), Ministerio de Ciencia, Innovación y Universidades and the Fondo Social Europeo (FSE) under grant RYC-2015-17239 (AEI/FSE, UE). The authors would like to thank the anonymous reviewers that helped to improve this paper through their valuable comments.Springer20192019info:eu-repo/semantics/articleinfo:eu-repo/semantics/acceptedVersionapplication/pdfapplication/pdfhttp://hdl.handle.net/10230/42897http://dx.doi.org/10.1007/s10846-019-01100-3reponame:Recercat. Dipósit de la Recerca de Catalunyainstname:Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya)InglésJournal of intelligent & robotic systems. 2019 Nov 16:1-11© Springer The final publication is available at Springer via https://doi.org/10.1007/s10846-019-01100-3info:eu-repo/semantics/openAccessoai:recercat.cat:10230/428972026-05-29T05:05:01Z |
| dc.title.none.fl_str_mv |
Part-of-speech and prosody-based approaches for robot speech and gesture synchronization |
| title |
Part-of-speech and prosody-based approaches for robot speech and gesture synchronization |
| spellingShingle |
Part-of-speech and prosody-based approaches for robot speech and gesture synchronization Pérez Mayos, Laura Human-computer interaction Multimodal interaction Humanoid robots Prosody Speech Gesture modelling Arm gesture synthesis Speech and gesture synchronization Text-to-gesture |
| title_short |
Part-of-speech and prosody-based approaches for robot speech and gesture synchronization |
| title_full |
Part-of-speech and prosody-based approaches for robot speech and gesture synchronization |
| title_fullStr |
Part-of-speech and prosody-based approaches for robot speech and gesture synchronization |
| title_full_unstemmed |
Part-of-speech and prosody-based approaches for robot speech and gesture synchronization |
| title_sort |
Part-of-speech and prosody-based approaches for robot speech and gesture synchronization |
| dc.creator.none.fl_str_mv |
Pérez Mayos, Laura Farrús, Mireia Adell, Jordi |
| author |
Pérez Mayos, Laura |
| author_facet |
Pérez Mayos, Laura Farrús, Mireia Adell, Jordi |
| author_role |
author |
| author2 |
Farrús, Mireia Adell, Jordi |
| author2_role |
author author |
| dc.subject.none.fl_str_mv |
Human-computer interaction Multimodal interaction Humanoid robots Prosody Speech Gesture modelling Arm gesture synthesis Speech and gesture synchronization Text-to-gesture |
| topic |
Human-computer interaction Multimodal interaction Humanoid robots Prosody Speech Gesture modelling Arm gesture synthesis Speech and gesture synchronization Text-to-gesture |
| description |
Humanoid robots are already among us and they are beginning to assume more social and personal roles, like guiding and assisting people. Thus, they should interact in a human-friendly manner, using not only verbal cues but also synchronized non-verbal and para-verbal cues. However, available robots are not able to communicate in this multimodal way, being just able to perform predefined gesture sequences, hand- crafted to accompany specific utterances. In the current paper, we propose a model based on three different approaches to extend humanoid robots communication behaviour with upper body gestures synchronized with the speech for novel utterances, exploiting part-of-speech grammatical information, prosody cues, and a combination of both. User studies confirm that our methods are able to produce natural, appropriate and good timed gesture sequences synchronized with speech, using both beat and emblematic gestures. |
| publishDate |
2019 |
| dc.date.none.fl_str_mv |
2019 2019 |
| dc.type.none.fl_str_mv |
info:eu-repo/semantics/article info:eu-repo/semantics/acceptedVersion |
| format |
article |
| status_str |
acceptedVersion |
| dc.identifier.none.fl_str_mv |
http://hdl.handle.net/10230/42897 http://dx.doi.org/10.1007/s10846-019-01100-3 |
| url |
http://hdl.handle.net/10230/42897 http://dx.doi.org/10.1007/s10846-019-01100-3 |
| dc.language.none.fl_str_mv |
Inglés |
| language_invalid_str_mv |
Inglés |
| dc.relation.none.fl_str_mv |
Journal of intelligent & robotic systems. 2019 Nov 16:1-11 |
| dc.rights.none.fl_str_mv |
info:eu-repo/semantics/openAccess |
| eu_rights_str_mv |
openAccess |
| dc.format.none.fl_str_mv |
application/pdf application/pdf |
| dc.publisher.none.fl_str_mv |
Springer |
| publisher.none.fl_str_mv |
Springer |
| dc.source.none.fl_str_mv |
reponame:Recercat. Dipósit de la Recerca de Catalunya instname:Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya) |
| instname_str |
Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya) |
| reponame_str |
Recercat. Dipósit de la Recerca de Catalunya |
| collection |
Recercat. Dipósit de la Recerca de Catalunya |
| repository.name.fl_str_mv |
|
| repository.mail.fl_str_mv |
|
| _version_ |
1869403463055572992 |
| score |
15,812455 |