Solving a markov decision process multidimensional problem with tensor decomposition
Markov Decision Process (MDP) is a model used for planning decision-making of agents in stochastic and completely observable environments. Although much research is focused on solving atomic MDP problems in tabular forms or MDP problems with factored representations, none is based on tensor decompos...
| Author: | |
|---|---|
| Format: | master thesis |
| Status: | Published version |
| Publication Date: | 2021 |
| Country: | Brasil |
| Institution: | Pontifícia Universidade Católica do Rio Grande do Sul (PUCRS) |
| Repository: | Biblioteca Digital de Teses e Dissertações da PUC_RS |
| Language: | English |
| OAI Identifier: | oai:tede2.pucrs.br:tede/9832 |
| Online Access: | http://tede2.pucrs.br/tede2/handle/tede/9832 |
| Access Level: | Open access |
| Keyword: | Artificial Intelligence CANDECOMP-PARAFAC Decomposition Markov Decision Process Tensor Algebra Tensor Decomposition Inteligência Artificial Decomposição CANDECOMP-PARAFAC Processo de ´Decisão de Markov Ágebra Tensorial Decomposição de Tensores CIENCIA DA COMPUTACAO::TEORIA DA COMPUTACAO |
| Summary: | Markov Decision Process (MDP) is a model used for planning decision-making of agents in stochastic and completely observable environments. Although much research is focused on solving atomic MDP problems in tabular forms or MDP problems with factored representations, none is based on tensor decomposition methods. Solving MDPs using tensor algebra offers the prospect of leveraging advances in tensor-based calculations to increase MDP solvers’ efficiency. In this research, first, we formalize MDP multidimensional problems using tensor algebra. Second, we develop an MDP solver using the CANDECOMP-PARAFAC tensor decomposition method to compact state transition matrices. The solver uses the value iteration and policy iteration algorithms to compute the solution compactly. Then, we empirically evaluate the compact algorithms compared to tabular methods. As a result, we show that the tensor approach can compute larger problems using substantially less memory, opening up new possibilities for tensor-based methods for stochastic planning. |
|---|