Dynamic tracking for object pose prediction in autonomous robots
Unleashing autonomous robots social capabilities can is the key feature to guarantee their smooth integration in crowded environments. To achieve social awareness, AMRs need to perceive, understand and actuate to visual and range stimuli. During the past years, multiple studies have leveraged the us...
| Autor: | |
|---|---|
| Tipo de documento: | dissertação |
| Data de publicação: | 2024 |
| País: | España |
| Recursos: | Universitat Politècnica de Catalunya (UPC) |
| Repositório: | UPCommons. Portal del coneixement obert de la UPC |
| Idioma: | inglês |
| OAI Identifier: | oai:upcommons.upc.edu:2117/411729 |
| Acesso em linha: | https://hdl.handle.net/2117/411729 |
| Access Level: | Acceso aberto |
| Palavra-chave: | Computer vision Image processing Robot vision Autonomous robots Visió per ordinador Imatges--Processament Visió artificial (Robòtica) Robots autònoms Àrees temàtiques de la UPC::Informàtica |
| Resumo: | Unleashing autonomous robots social capabilities can is the key feature to guarantee their smooth integration in crowded environments. To achieve social awareness, AMRs need to perceive, understand and actuate to visual and range stimuli. During the past years, multiple studies have leveraged the use of LiDAR-based solutions to identify dynamic objects in the 3D space. However, in applications where budget is a matter, robots might be required to equip other extereoceptive sensors. This project proposes a real-time multiple person detection and tracking framework for indoor autonomous mobile robots equipped with a stereo camera. The outcome of the tracking system is to predict human poses, orientations and velocities for later object avoidance and path planning redefinition tasks. While the framework adopts a multimodal approach, a brief overview of an initial tracking-by-detection proposal based on disparity maps is also given. Despite its generic architecture, a particular study case based on Yuman’s Buddy has been taken as reference. For the sake of the available technology, a ZED2 stereo camera is used to get both dense 3D point-clouds and 2D RGB images. Thus, the first half of the study delves into the selection and integration of 2 real-time deep-learning-based object detectors, whereas the second half merges the novel JRMOT tracking system with both custom detectors under a ROS2 workspace. Since the output of the framework is a 3D pose estimation, all data is referred to the camera frame. Later transformations (e.g. to the base frame) can be done depending on each robot architecture. Half the tests performed along this thesis are based on pure ROS2 bag files deliberately recorded for the use-case scenarios that an indoor AMR can encounter. This is, concretely, crowded hospital corridors with potential partial or fully body occlusions. The study aims to evaluate the real-time performance (+30FPS) of the selected 2D detectors and the whole framework based on the input image and pointcloud publishing rates. Besides, an evaluation on the precision, accuracy and ID switches is contemplated using the novel benchmark JRDB. While results resulted satisfactory in terms of the 2D detectors accuracy (∼ 98%), runtime tests did not performed in real-time. In addition, the evaluation of the data associator and pose filters was finally conditioned by the 3D detector model weights. |
|---|