Assisting the training of deep neural networks with applications to computer vision
[eng] Deep learning has recently been enjoying an increasing popularity due to its success in solving challenging tasks. In particular, deep learning has proven to be effective in a large variety of computer vision tasks, such as image classification, object recognition and image parsing. Contrary t...
| Autor: | |
|---|---|
| Formato: | tesis doctoral |
| Fecha de publicación: | 2015 |
| País: | España |
| Recursos: | Universidad de Barcelona |
| Repositorio: | Dipòsit Digital de la UB |
| OAI Identifier: | oai:diposit.ub.edu:2445/67591 |
| Acesso em linha: | https://hdl.handle.net/2445/67591 http://tdx.cat/handle/10803/316577 |
| Access Level: | acceso abierto |
| Palavra-chave: | Aprenentatge profund Visió per ordinador Teledetecció Processament d'imatges Deep learning Computer vision Remote sensing Image processing |
| id |
ES_618d50aba34436f18cc6cdad4dcb9096 |
|---|---|
| oai_identifier_str |
oai:diposit.ub.edu:2445/67591 |
| network_acronym_str |
ES |
| network_name_str |
España |
| repository_id_str |
|
| spelling |
Assisting the training of deep neural networks with applications to computer visionRomero, AdrianaAprenentatge profundVisió per ordinadorTeledeteccióProcessament d'imatgesDeep learningComputer visionRemote sensingImage processing[eng] Deep learning has recently been enjoying an increasing popularity due to its success in solving challenging tasks. In particular, deep learning has proven to be effective in a large variety of computer vision tasks, such as image classification, object recognition and image parsing. Contrary to previous research, which required engineered feature representations, designed by experts, in order to succeed, deep learning attempts to learn representation hierarchies automatically from data. More recently, the trend has been to go deeper with representation hierarchies. Learning (very) deep representation hierarchies is a challenging task, which involves the optimization of highly non- convex functions. Therefore, the search for algorithms to ease the learning of (very) deep representation hierarchies from data is extensive and ongoing. In this thesis, we tackle the challenging problem of easing the learning of (very) deep representation hierarchies. We present a hyper-parameter free, off-the-shelf, simple and fast unsupervised algorithm to discover hidden structure from the input data by enforcing a very strong form of sparsity. We study the applicability and potential of the algorithm to learn representations of varying depth in a handful of applications and domains, highlighting the ability of the algorithm to provide discriminative feature representations that are able to achieve top performance. Yet, while emphasizing the great value of unsupervised learning methods when labeled data is scarce, the recent industrial success of deep learning has revolved around supervised learning. Supervised learning is currently the focus of many recent research advances, which have shown to excel at many computer vision tasks. Top performing systems often involve very large and deep models, which are not well suited for applications with time or memory limitations. More in line with the current trends, we engage in making top performing models more efficient, by designing very deep and thin models. Since training such very deep models still appears to be a challenging task, we introduce a novel algorithm that guides the training of very thin and deep models by hinting their intermediate representations. Very deep and thin models trained by the proposed algorithm end up extracting feature representations that are comparable or even better performing than the ones extracted by large state-of-the-art models, while compellingly reducing the time and memory consumption of the model.Universitat de BarcelonaGatta, CarloRadeva, Petia2015info:eu-repo/semantics/doctoralThesisapplication/pdfhttps://hdl.handle.net/2445/67591http://tdx.cat/handle/10803/316577Tesis Doctorals - Departament - Matemàtica Aplicada i Anàlisireponame:Dipòsit Digital de la UBinstname:Universidad de BarcelonaIngléscc by-nc-sa (c) Romero, 2015http://creativecommons.org/licenses/by-nc-sa/3.0/es/info:eu-repo/semantics/openAccessoai:diposit.ub.edu:2445/675912026-05-27T06:46:51Z |
| dc.title.none.fl_str_mv |
Assisting the training of deep neural networks with applications to computer vision |
| title |
Assisting the training of deep neural networks with applications to computer vision |
| spellingShingle |
Assisting the training of deep neural networks with applications to computer vision Romero, Adriana Aprenentatge profund Visió per ordinador Teledetecció Processament d'imatges Deep learning Computer vision Remote sensing Image processing |
| title_short |
Assisting the training of deep neural networks with applications to computer vision |
| title_full |
Assisting the training of deep neural networks with applications to computer vision |
| title_fullStr |
Assisting the training of deep neural networks with applications to computer vision |
| title_full_unstemmed |
Assisting the training of deep neural networks with applications to computer vision |
| title_sort |
Assisting the training of deep neural networks with applications to computer vision |
| dc.creator.none.fl_str_mv |
Romero, Adriana |
| author |
Romero, Adriana |
| author_facet |
Romero, Adriana |
| author_role |
author |
| dc.contributor.none.fl_str_mv |
Gatta, Carlo Radeva, Petia |
| dc.subject.none.fl_str_mv |
Aprenentatge profund Visió per ordinador Teledetecció Processament d'imatges Deep learning Computer vision Remote sensing Image processing |
| topic |
Aprenentatge profund Visió per ordinador Teledetecció Processament d'imatges Deep learning Computer vision Remote sensing Image processing |
| description |
[eng] Deep learning has recently been enjoying an increasing popularity due to its success in solving challenging tasks. In particular, deep learning has proven to be effective in a large variety of computer vision tasks, such as image classification, object recognition and image parsing. Contrary to previous research, which required engineered feature representations, designed by experts, in order to succeed, deep learning attempts to learn representation hierarchies automatically from data. More recently, the trend has been to go deeper with representation hierarchies. Learning (very) deep representation hierarchies is a challenging task, which involves the optimization of highly non- convex functions. Therefore, the search for algorithms to ease the learning of (very) deep representation hierarchies from data is extensive and ongoing. In this thesis, we tackle the challenging problem of easing the learning of (very) deep representation hierarchies. We present a hyper-parameter free, off-the-shelf, simple and fast unsupervised algorithm to discover hidden structure from the input data by enforcing a very strong form of sparsity. We study the applicability and potential of the algorithm to learn representations of varying depth in a handful of applications and domains, highlighting the ability of the algorithm to provide discriminative feature representations that are able to achieve top performance. Yet, while emphasizing the great value of unsupervised learning methods when labeled data is scarce, the recent industrial success of deep learning has revolved around supervised learning. Supervised learning is currently the focus of many recent research advances, which have shown to excel at many computer vision tasks. Top performing systems often involve very large and deep models, which are not well suited for applications with time or memory limitations. More in line with the current trends, we engage in making top performing models more efficient, by designing very deep and thin models. Since training such very deep models still appears to be a challenging task, we introduce a novel algorithm that guides the training of very thin and deep models by hinting their intermediate representations. Very deep and thin models trained by the proposed algorithm end up extracting feature representations that are comparable or even better performing than the ones extracted by large state-of-the-art models, while compellingly reducing the time and memory consumption of the model. |
| publishDate |
2015 |
| dc.date.none.fl_str_mv |
2015 |
| dc.type.none.fl_str_mv |
info:eu-repo/semantics/doctoralThesis |
| format |
doctoralThesis |
| dc.identifier.none.fl_str_mv |
https://hdl.handle.net/2445/67591 http://tdx.cat/handle/10803/316577 |
| url |
https://hdl.handle.net/2445/67591 http://tdx.cat/handle/10803/316577 |
| dc.language.none.fl_str_mv |
Inglés |
| language_invalid_str_mv |
Inglés |
| dc.rights.none.fl_str_mv |
cc by-nc-sa (c) Romero, 2015 http://creativecommons.org/licenses/by-nc-sa/3.0/es/ info:eu-repo/semantics/openAccess |
| rights_invalid_str_mv |
cc by-nc-sa (c) Romero, 2015 http://creativecommons.org/licenses/by-nc-sa/3.0/es/ |
| eu_rights_str_mv |
openAccess |
| dc.format.none.fl_str_mv |
application/pdf |
| dc.publisher.none.fl_str_mv |
Universitat de Barcelona |
| publisher.none.fl_str_mv |
Universitat de Barcelona |
| dc.source.none.fl_str_mv |
Tesis Doctorals - Departament - Matemàtica Aplicada i Anàlisi reponame:Dipòsit Digital de la UB instname:Universidad de Barcelona |
| instname_str |
Universidad de Barcelona |
| reponame_str |
Dipòsit Digital de la UB |
| collection |
Dipòsit Digital de la UB |
| repository.name.fl_str_mv |
|
| repository.mail.fl_str_mv |
|
| _version_ |
1869409424981884928 |
| score |
15,228081 |