PBNS: physically based neural simulation for unsupervised garment pose space deformation

We present a methodology to automatically obtain Pose Space Deformation (PSD) basis for rigged garments through deep learning. Classical approaches rely on Physically Based Simulations (PBS) to animate clothes. These are general solutions that, given a sufficiently fine-grained discretization of spa...

ver descrição completa

Detalhes bibliográficos
Autores: Bertiche, Hugo, Madadi, Meysam, Escalera Guerrero, Sergio
Formato: artículo
Estado:Versión aceptada para publicación
Fecha de publicación:2021
País:España
Recursos:Universidad de Barcelona
Repositorio:Dipòsit Digital de la UB
OAI Identifier:oai:diposit.ub.edu:2445/190925
Acesso em linha:https://hdl.handle.net/2445/190925
Access Level:acceso abierto
Palavra-chave:Aprenentatge automàtic
Visió per ordinador
Simulació per ordinador
Xarxes neuronals (Informàtica)
Machine learning
Computer vision
Computer simulation
Neural networks (Computer science)
id ES_fb1a59d7a4fcd00e919770b16d357a8e
oai_identifier_str oai:diposit.ub.edu:2445/190925
network_acronym_str ES
network_name_str España
repository_id_str
spelling PBNS: physically based neural simulation for unsupervised garment pose space deformationBertiche, HugoMadadi, MeysamEscalera Guerrero, SergioAprenentatge automàticVisió per ordinadorSimulació per ordinadorXarxes neuronals (Informàtica)Machine learningComputer visionComputer simulationNeural networks (Computer science)We present a methodology to automatically obtain Pose Space Deformation (PSD) basis for rigged garments through deep learning. Classical approaches rely on Physically Based Simulations (PBS) to animate clothes. These are general solutions that, given a sufficiently fine-grained discretization of space and time, can achieve highly realistic results. However, they are computationally expensive and any scene modification prompts the need of re-simulation. Linear Blend Skinning (LBS) with PSD offers a lightweight alternative to PBS, though, it needs huge volumes of data to learn proper PSD. We propose using deep learning, formulated as an implicit PBS, to un-supervisedly learn realistic cloth Pose Space Deformations in a constrained scenario: dressed humans. Furthermore, we show it is possible to train these models in an amount of time comparable to a PBS of a few sequences. To the best of our knowledge, we are the first to propose a neural simulator for cloth. While deep-based approaches in the domain are becoming a trend, these are data-hungry models. Moreover, authors often propose complex formulations to better learn wrinkles from PBS data. Supervised learning leads to physically inconsistent predictions that require collision solving to be used. Also, dependency on PBS data limits the scalability of these solutions, while their formulation hinders its applicability and compatibility. By proposing an unsupervised methodology to learn PSD for LBS models (3D animation standard), we overcome both of these drawbacks. Results obtained show cloth-consistency in the animated garments and meaningful pose-dependant folds and wrinkles. Our solution is extremely efficient, handles multiple layers of cloth, allows unsupervised outfit resizing and can be easily applied to any custom 3D avatar.Association for Computing Machinery2021info:eu-repo/semantics/articleinfo:eu-repo/semantics/acceptedVersionapplication/pdfhttps://hdl.handle.net/2445/190925Articles publicats en revistes (Matemàtiques i Informàtica)reponame:Dipòsit Digital de la UBinstname:Universidad de BarcelonaInglésVersió postprint del document publicat a: https://doi.org/10.1145/3478513.3480479ACM Transactions on Graphics, 2021, vol. 40, num. 6, p. 1-14https://doi.org/10.1145/3478513.3480479(c) Association for Computing Machinery, 2021info:eu-repo/semantics/openAccessoai:diposit.ub.edu:2445/1909252026-05-27T06:46:51Z
dc.title.none.fl_str_mv PBNS: physically based neural simulation for unsupervised garment pose space deformation
title PBNS: physically based neural simulation for unsupervised garment pose space deformation
spellingShingle PBNS: physically based neural simulation for unsupervised garment pose space deformation
Bertiche, Hugo
Aprenentatge automàtic
Visió per ordinador
Simulació per ordinador
Xarxes neuronals (Informàtica)
Machine learning
Computer vision
Computer simulation
Neural networks (Computer science)
title_short PBNS: physically based neural simulation for unsupervised garment pose space deformation
title_full PBNS: physically based neural simulation for unsupervised garment pose space deformation
title_fullStr PBNS: physically based neural simulation for unsupervised garment pose space deformation
title_full_unstemmed PBNS: physically based neural simulation for unsupervised garment pose space deformation
title_sort PBNS: physically based neural simulation for unsupervised garment pose space deformation
dc.creator.none.fl_str_mv Bertiche, Hugo
Madadi, Meysam
Escalera Guerrero, Sergio
author Bertiche, Hugo
author_facet Bertiche, Hugo
Madadi, Meysam
Escalera Guerrero, Sergio
author_role author
author2 Madadi, Meysam
Escalera Guerrero, Sergio
author2_role author
author
dc.subject.none.fl_str_mv Aprenentatge automàtic
Visió per ordinador
Simulació per ordinador
Xarxes neuronals (Informàtica)
Machine learning
Computer vision
Computer simulation
Neural networks (Computer science)
topic Aprenentatge automàtic
Visió per ordinador
Simulació per ordinador
Xarxes neuronals (Informàtica)
Machine learning
Computer vision
Computer simulation
Neural networks (Computer science)
description We present a methodology to automatically obtain Pose Space Deformation (PSD) basis for rigged garments through deep learning. Classical approaches rely on Physically Based Simulations (PBS) to animate clothes. These are general solutions that, given a sufficiently fine-grained discretization of space and time, can achieve highly realistic results. However, they are computationally expensive and any scene modification prompts the need of re-simulation. Linear Blend Skinning (LBS) with PSD offers a lightweight alternative to PBS, though, it needs huge volumes of data to learn proper PSD. We propose using deep learning, formulated as an implicit PBS, to un-supervisedly learn realistic cloth Pose Space Deformations in a constrained scenario: dressed humans. Furthermore, we show it is possible to train these models in an amount of time comparable to a PBS of a few sequences. To the best of our knowledge, we are the first to propose a neural simulator for cloth. While deep-based approaches in the domain are becoming a trend, these are data-hungry models. Moreover, authors often propose complex formulations to better learn wrinkles from PBS data. Supervised learning leads to physically inconsistent predictions that require collision solving to be used. Also, dependency on PBS data limits the scalability of these solutions, while their formulation hinders its applicability and compatibility. By proposing an unsupervised methodology to learn PSD for LBS models (3D animation standard), we overcome both of these drawbacks. Results obtained show cloth-consistency in the animated garments and meaningful pose-dependant folds and wrinkles. Our solution is extremely efficient, handles multiple layers of cloth, allows unsupervised outfit resizing and can be easily applied to any custom 3D avatar.
publishDate 2021
dc.date.none.fl_str_mv 2021
dc.type.none.fl_str_mv info:eu-repo/semantics/article
info:eu-repo/semantics/acceptedVersion
format article
status_str acceptedVersion
dc.identifier.none.fl_str_mv https://hdl.handle.net/2445/190925
url https://hdl.handle.net/2445/190925
dc.language.none.fl_str_mv Inglés
language_invalid_str_mv Inglés
dc.relation.none.fl_str_mv Versió postprint del document publicat a: https://doi.org/10.1145/3478513.3480479
ACM Transactions on Graphics, 2021, vol. 40, num. 6, p. 1-14
https://doi.org/10.1145/3478513.3480479
dc.rights.none.fl_str_mv (c) Association for Computing Machinery, 2021
info:eu-repo/semantics/openAccess
rights_invalid_str_mv (c) Association for Computing Machinery, 2021
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv Association for Computing Machinery
publisher.none.fl_str_mv Association for Computing Machinery
dc.source.none.fl_str_mv Articles publicats en revistes (Matemàtiques i Informàtica)
reponame:Dipòsit Digital de la UB
instname:Universidad de Barcelona
instname_str Universidad de Barcelona
reponame_str Dipòsit Digital de la UB
collection Dipòsit Digital de la UB
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869425298265604096
score 15.301629

Registos relacionados