Human-like systematic generalization through a meta-learning neural network

The power of human language and thought arises from systematic compositionality—the algebraic ability to understand and produce novel combinations from known components. Fodor and Pylyshyn1 famously argued that artificial neural networks lack this capacity and are therefore not viable models of the...

Descripción completa

Detalles Bibliográficos
Autores: Baroni, Marco, Lake, Brenden M.
Tipo de recurso: artículo
Estado:Versión aceptada para publicación
Fecha de publicación:2023
País:España
Institución:Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya)
Repositorio:Recercat. Dipósit de la Recerca de Catalunya
OAI Identifier:oai:recercat.cat:10230/58561
Acceso en línea:http://hdl.handle.net/10230/58561
http://dx.doi.org/10.1038/s41586-023-06668-3
Access Level:acceso abierto
Palabra clave:Xarxes neuronals (Informàtica)
Metaaprenentatge
Neurociències
id ES_6b4f3fd7e15196dc38c0d292fd5ddef7
oai_identifier_str oai:recercat.cat:10230/58561
network_acronym_str ES
network_name_str España
repository_id_str
spelling Human-like systematic generalization through a meta-learning neural networkBaroni, MarcoLake, Brenden M.Xarxes neuronals (Informàtica)MetaaprenentatgeNeurociènciesThe power of human language and thought arises from systematic compositionality—the algebraic ability to understand and produce novel combinations from known components. Fodor and Pylyshyn1 famously argued that artificial neural networks lack this capacity and are therefore not viable models of the mind. Neural networks have advanced considerably in the years since, yet the systematicity challenge persists. Here we successfully address Fodor and Pylyshyn’s challenge by providing evidence that neural networks can achieve human-like systematicity when optimized for their compositional skills. To do so, we introduce the meta-learning for compositionality (MLC) approach for guiding training through a dynamic stream of compositional tasks. To compare humans and machines, we conducted human behavioural experiments using an instruction learning paradigm. After considering seven different models, we found that, in contrast to perfectly systematic but rigid probabilistic symbolic models, and perfectly flexible but unsystematic neural networks, only MLC achieves both the systematicity and flexibility needed for human-like generalization. MLC also advances the compositional skills of machine learning systems in several systematic generalization benchmarks. Our results show how a standard neural network architecture, optimized for its compositional skills, can mimic human systematic generalization in a head-to-head comparison.Nature Research202320232023info:eu-repo/semantics/articleinfo:eu-repo/semantics/acceptedVersionapplication/pdfapplication/pdfhttp://hdl.handle.net/10230/58561http://dx.doi.org/10.1038/s41586-023-06668-3reponame:Recercat. Dipósit de la Recerca de Catalunyainstname:Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya)InglésNature. 2023 Nov;623(7985):115–21.© The Author(s) 2023. This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.http://creativecommons.org/licenses/by/4.0/info:eu-repo/semantics/openAccessoai:recercat.cat:10230/585612026-05-29T05:05:01Z
dc.title.none.fl_str_mv Human-like systematic generalization through a meta-learning neural network
title Human-like systematic generalization through a meta-learning neural network
spellingShingle Human-like systematic generalization through a meta-learning neural network
Baroni, Marco
Xarxes neuronals (Informàtica)
Metaaprenentatge
Neurociències
title_short Human-like systematic generalization through a meta-learning neural network
title_full Human-like systematic generalization through a meta-learning neural network
title_fullStr Human-like systematic generalization through a meta-learning neural network
title_full_unstemmed Human-like systematic generalization through a meta-learning neural network
title_sort Human-like systematic generalization through a meta-learning neural network
dc.creator.none.fl_str_mv Baroni, Marco
Lake, Brenden M.
author Baroni, Marco
author_facet Baroni, Marco
Lake, Brenden M.
author_role author
author2 Lake, Brenden M.
author2_role author
dc.subject.none.fl_str_mv Xarxes neuronals (Informàtica)
Metaaprenentatge
Neurociències
topic Xarxes neuronals (Informàtica)
Metaaprenentatge
Neurociències
description The power of human language and thought arises from systematic compositionality—the algebraic ability to understand and produce novel combinations from known components. Fodor and Pylyshyn1 famously argued that artificial neural networks lack this capacity and are therefore not viable models of the mind. Neural networks have advanced considerably in the years since, yet the systematicity challenge persists. Here we successfully address Fodor and Pylyshyn’s challenge by providing evidence that neural networks can achieve human-like systematicity when optimized for their compositional skills. To do so, we introduce the meta-learning for compositionality (MLC) approach for guiding training through a dynamic stream of compositional tasks. To compare humans and machines, we conducted human behavioural experiments using an instruction learning paradigm. After considering seven different models, we found that, in contrast to perfectly systematic but rigid probabilistic symbolic models, and perfectly flexible but unsystematic neural networks, only MLC achieves both the systematicity and flexibility needed for human-like generalization. MLC also advances the compositional skills of machine learning systems in several systematic generalization benchmarks. Our results show how a standard neural network architecture, optimized for its compositional skills, can mimic human systematic generalization in a head-to-head comparison.
publishDate 2023
dc.date.none.fl_str_mv 2023
2023
2023
dc.type.none.fl_str_mv info:eu-repo/semantics/article
info:eu-repo/semantics/acceptedVersion
format article
status_str acceptedVersion
dc.identifier.none.fl_str_mv http://hdl.handle.net/10230/58561
http://dx.doi.org/10.1038/s41586-023-06668-3
url http://hdl.handle.net/10230/58561
http://dx.doi.org/10.1038/s41586-023-06668-3
dc.language.none.fl_str_mv Inglés
language_invalid_str_mv Inglés
dc.relation.none.fl_str_mv Nature. 2023 Nov;623(7985):115–21.
dc.rights.none.fl_str_mv http://creativecommons.org/licenses/by/4.0/
info:eu-repo/semantics/openAccess
rights_invalid_str_mv http://creativecommons.org/licenses/by/4.0/
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
application/pdf
dc.publisher.none.fl_str_mv Nature Research
publisher.none.fl_str_mv Nature Research
dc.source.none.fl_str_mv reponame:Recercat. Dipósit de la Recerca de Catalunya
instname:Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya)
instname_str Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya)
reponame_str Recercat. Dipósit de la Recerca de Catalunya
collection Recercat. Dipósit de la Recerca de Catalunya
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869410186026811392
score 15.812455

Similares en LA Referencia