Revista UIS Ingenierías

Vol. 22 No. 4 (2023): Revista UIS Ingenierías
Articles

Tracking and Grasping of Moving Objects Using Aerial Robotic Manipulators: A Brief Survey

PDF
  • Yeyson Alejandro Becerra-Mora ,
  • Sebastián Soto-Gaona
Yeyson Alejandro Becerra-Mora
Corporación Unificada Nacional de Educación Superior
Sebastián Soto-Gaona
Corporación Unificada Nacional de Educación Superior
DOI: https://doi.org/10.18273/revuin.v22n4-2023011

Published 2023-11-28

Keywords

  • UAV,
  • grasping,
  • tracking,
  • aerial manipulator,
  • cooperation,
  • detection,
  • UAV fleet,
  • computer vision,
  • manipulation,
  • moving objects
    • ...More
    Less

How to Cite

Becerra-Mora , Y. A. ., & Soto-Gaona , S. . (2023). Tracking and Grasping of Moving Objects Using Aerial Robotic Manipulators: A Brief Survey. Revista UIS Ingenierías, 22(4), 115–128. https://doi.org/10.18273/revuin.v22n4-2023011

Copyright (c) 2023 Revista UIS Ingenierías

Creative Commons License

This work is licensed under a Creative Commons Attribution-NoDerivatives 4.0 International License.

Abstract

Unmanned Aerial Vehicles (UAV) has evolved in recent years, their features have changed to be more useful to the society, although some years ago the drones had been thought to be teleoperated by humans and to take some pictures from above, which is useful; nevertheless, nowadays the drones are capable of developing autonomous tasks like tracking a dynamic target or even grasping different kind of objects. Some task like transporting heavy loads or manipulating complex shapes are more challenging for a single UAV, but for a fleet of them might be easier. This brief survey presents a compilation of relevant works related to tracking and grasping with aerial robotic manipulators, as well as cooperation among them. Moreover, challenges and limitations are presented in order to contribute with new areas of research. Finally, some trends in aerial manipulation are foreseeing for different sectors and relevant features for these kind of systems are standing out.

PDF

Downloads

Download data is not yet available.

References

  1. X. Ding, P. Guo, K. Xu, Y. Yu, “A review of aerial manipulation of small-scale rotorcraft unmanned robotic systems,” Chinese J. Aeronaut., vol. 32, no. 1, pp. 200–214, Jan. 2019, doi: https://doi.org/10.1016/j.cja.2018.05.012
  2. A. Ollero, M. Tognon, A. Suarez, D. Lee, A. Franchi, “Past, Present, and Future of Aerial Robotic Manipulators,” IEEE Trans. Robot., vol. 38, no. 1, pp. 626–645, 2022, doi: https://doi.org/10.1109/TRO.2021.3084395
  3. A. Suarez, R. Salmoral, P. J. Zarco-Perinan, A. Ollero, “Experimental Evaluation of Aerial Manipulation Robot in Contact With 15 kV Power Line: Shielded and Long Reach Configurations,” IEEE Access, vol. 9, pp. 94573–94585, 2021, doi: https://doi.org/10.1109/ACCESS.2021.3093856
  4. A. E. Jimenez-Cano, J. Martin, G. Heredia, A. Ollero, R. Cano, “Control of an aerial robot with multi-link arm for assembly tasks,” in 2013 IEEE International Conference on Robotics and Automation, 2013, pp. 4916–4921, doi: https://doi.org/10.1109/ICRA.2013.6631279
  5. A. Suarez, A. Caballero, A. Garofano, P. J. Sanchez-Cuevas, G. Heredia, and A. Ollero, “Aerial Manipulator With Rolling Base for Inspection of Pipe Arrays,” IEEE Access, vol. 8, pp. 162516–162532, 2020, doi: https://doi.org/10.1109/ACCESS.2020.3021126
  6. M. Polic, A. Ivanovic, B. Maric, B. Arbanas, J. Tabak, and M. Orsag, “Structured Ecological Cultivation with Autonomous Robots in Indoor Agriculture,” in 2021 16th International Conference on Telecommunications (ConTEL), 2021, pp. 189–195, doi: https://doi.org/10.23919/ConTEL52528.2021.9495963
  7. A. Gawel et al., “Aerial picking and delivery of magnetic objects with MAVs,” in 2017 IEEE International Conference on Robotics and Automation (ICRA), 2017, pp. 5746–5752, doi: https://doi.org/10.1109/ICRA.2017.7989675
  8. D. Lee, H. Seo, I. Jang, S. J. Lee, and H. J. Kim, “Aerial Manipulator Pushing a Movable Structure Using a DOB-Based Robust Controller,” IEEE Robot. Autom. Lett., vol. 6, no. 2, pp. 723–730, 2021, doi: https://doi.org/10.1109/LRA.2020.3047779
  9. P. K. Allen, A. Timcenko, B. Yoshimi, and P. Michelman, “Automated tracking and grasping of a moving object with a robotic hand-eye system,” IEEE Trans. Robot. Autom., vol. 9, no. 2, pp. 152–165, Apr. 1993, doi: https://doi.org/10.1109/70.238279
  10. N. P. Papanikolopoulos, P. K. Khosla, T. Kanade, “Visual tracking of a moving target by a camera mounted on a robot: a combination of control and vision,” IEEE Trans. Robot. Autom., vol. 9, no. 1, pp. 14–35, 1993, doi: https://doi.org/10.1109/70.210792
  11. M. Shibata and N. Kobayashi, “Image-based visual tracking for moving targets with active stereo vision robot,” in 2006 SICE-ICASE International Joint Conference, 2006, pp. 5329–5334, doi: https://doi.org/10.1109/SICE.2006.315320
  12. I. S. Shin, S.-H. Nam, R. G. Roberts, S. B. Moon, “Minimum-Time Algorithm For Intercepting An Object On The Conveyor Belt By Robot,” in 2007 International Symposium on Computational Intelligence in Robotics and Automation, Jun. 2007, pp. 362–367, doi: https://doi.org/10.1109/CIRA.2007.382906
  13. L. M. Belmonte, R. Morales, A. Fernández-Caballero, “Computer Vision in Autonomous Unmanned Aerial Vehicles—A Systematic Mapping Study,” Appl. Sci., vol. 9, no. 15, p. 3196, 2019, doi: https://doi.org/10.3390/app9153196
  14. K. Bodie et al., “Active Interaction Force Control for Contact-Based Inspection With a Fully Actuated Aerial Vehicle,” IEEE Trans. Robot., vol. 37, no. 3, pp. 709–722, Jun. 2021, doi: https://doi.org/10.1109/TRO.2020.3036623
  15. K. Kondak et al., “Aerial manipulation robot composed of an autonomous helicopter and a 7 degrees of freedom industrial manipulator,” in 2014 IEEE International Conference on Robotics and Automation (ICRA), May 2014, pp. 2107–2112, doi: https://doi.org/10.1109/ICRA.2014.6907148
  16. B. Gabrich, D. Saldana, V. Kumar, M. Yim, “A Flying Gripper Based on Cuboid Modular Robots,” in 2018 IEEE International Conference on Robotics and Automation (ICRA), May 2018, pp. 7024–7030, doi: https://doi.org/10.1109/ICRA.2018.8460682
  17. F. Caccavale, G. Giglio, G. Muscio, F. Pierri, “Adaptive control for UAVs equipped with a robotic arm,” IFAC Proc. Vol., vol. 47, no. 3, pp. 11049–11054, 2014, doi: https://doi.org/10.3182/20140824-6-ZA-1003.00790
  18. M. Laiacker, F. Huber, K. Kondak, “High accuracy visual servoing for aerial manipulation using a 7 degrees of freedom industrial manipulator,” in 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Oct. 2016, pp. 1631–1636, doi: https://doi.org/10.1109/IROS.2016.7759263
  19. K. Gkountas and A. Tzes, “Leader/Follower Force Control of Aerial Manipulators,” IEEE Access, vol. 9, pp. 17584–17595, 2021, doi: https://doi.org/10.1109/ACCESS.2021.3053654
  20. V. Lippiello and F. Ruggiero, “Cartesian Impedance Control of a UAV with a Robotic Arm,” IFAC Proc. Vol., vol. 45, no. 22, pp. 704–709, 2012, doi: https://doi.org/10.3182/20120905-3-HR-2030.00158
  21. R. Mo, H. Cai, and S. L. Dai, “Unit Quaternion Based Attitude Control of An Aerial Manipulator,” IFAC-PapersOnLine, vol. 52, no. 24, pp. 190–194, 2019, doi: https://doi.org/10.1016/j.ifacol.2019.12.405
  22. R. Naldi, A. Macchelli, N. Mimmo, and L. Marconi, “Robust Control of an Aerial Manipulator Interacting with the Environment,” IFAC-PapersOnLine, vol. 51, no. 13, pp. 537–542, 2018, doi: https://doi.org/10.1016/j.ifacol.2018.07.335
  23. P. D. Suthar and V. Sangwan, “Contact Force-Velocity Control for a Planar Aerial Manipulator,” IFAC-PapersOnLine, vol. 55, no. 1, pp. 1–7, 2022, doi: https://doi.org/10.1016/j.ifacol.2022.04.001
  24. S. Kim, H. Seo, J. Shin, H. J. Kim, “Cooperative Aerial Manipulation Using Multirotors With Multi-DOF Robotic Arms,” IEEE/ASME Trans. Mechatronics, vol. 23, no. 2, pp. 702–713, 2018, doi: https://doi.org/10.1109/TMECH.2018.2792318
  25. Y. Chen et al., “Robust Control for Unmanned Aerial Manipulator Under Disturbances,” IEEE Access, vol. 8, pp. 129869–129877, 2020, doi: https://doi.org/10.1109/ACCESS.2020.3008971
  26. F. Quan, H. Chen, Y. Li, Y. Lou, J. Chen, and Y. Liu, “Singularity-Robust Hybrid Visual Servoing Control for Aerial Manipulator,” in 2018 IEEE International Conference on Robotics and Biomimetics (ROBIO), Dec. 2018, pp. 562–568, doi: https://doi.org/10.1109/ROBIO.2018.8665260
  27. J. E. Gomez-Balderas, G. Flores, L. R. García Carrillo, and R. Lozano, “Tracking a Ground Moving Target with a Quadrotor Using Switching Control,” J. Intell. Robot. Syst., vol. 70, no. 1–4, pp. 65–78, 2013, doi: https://doi.org/10.1007/s10846-012-9747-9
  28. C. Teuliere, L. Eck, and E. Marchand, “Chasing a moving target from a flying UAV,” in 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems, Sep. 2011, pp. 4929–4934, doi: https://doi.org/10.1109/IROS.2011.6094404
  29. J. Chen, T. Liu, S. Shen, “Tracking a moving target in cluttered environments using a quadrotor,” in 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Oct. 2016, pp. 446–453, doi: https://doi.org/10.1109/IROS.2016.7759092
  30. Y. Wu, Y. Sui, G. Wang, “Vision-Based Real-Time Aerial Object Localization and Tracking for UAV Sensing System,” IEEE Access, vol. 5, pp. 23969–23978, 2017, doi: https://doi.org/10.1109/ACCESS.2017.2764419
  31. M. Sepehri Movafegh, S. M. M. Dehghan, R. Zardashti, “Three-dimensional guidance and control for ground moving target tracking by a quadrotor,” Aeronaut. J., vol. 125, no. 1290, pp. 1380–1407, Aug. 2021, doi: https://doi.org/10.1017/aer.2021.23
  32. J. Li, D. H. Ye, T. Chung, M. Kolsch, J. Wachs, and C. Bouman, “Multi-target detection and tracking from a single camera in Unmanned Aerial Vehicles (UAVs),” in 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Oct. 2016, pp. 4992–4997, doi: https://doi.org/10.1109/IROS.2016.7759733
  33. B. Herissé, T. Hamel, R. Mahony, and F.-X. Russotto, “Landing a VTOL Unmanned Aerial Vehicle on a Moving Platform Using Optical Flow,” IEEE Trans. Robot., vol. 28, no. 1, pp. 77–89, 2012, doi: https://doi.org/10.1109/TRO.2011.2163435
  34. H. T. Zhang et al., “Visual Navigation and Landing Control of an Unmanned Aerial Vehicle on a Moving Autonomous Surface Vehicle via Adaptive Learning,” IEEE Trans. Neural Networks Learn. Syst., vol. 32, no. 12, pp. 5345–5355, 2021, doi: https://doi.org/10.1109/TNNLS.2021.3080980
  35. T. Baca et al., “Autonomous landing on a moving vehicle with an unmanned aerial vehicle,” J. F. Robot., vol. 36, no. 5, pp. 874–891, 2019, doi: https://doi.org/10.1002/rob.21858
  36. K. Guo, P. Tang, H. Wang, D. Lin, X. Cui, “Autonomous Landing of a Quadrotor on a Moving Platform via Model Predictive Control,” Aerospace, vol. 9, no. 1, p. 34, Jan. 2022, doi: https://doi.org/10.3390/aerospace9010034
  37. J. Xie, X. Peng, H. Wang, W. Niu, X. Zheng, “UAV Autonomous Tracking and Landing Based on Deep Reinforcement Learning Strategy,” Sensors, vol. 20, no. 19, p. 5630, Oct. 2020, doi: https://doi.org/10.3390/s20195630
  38. Y. Xu, Z. Liu, and X. Wang, “Monocular Vision based Autonomous Landing of Quadrotor through Deep Reinforcement Learning,” in 2018 37th Chinese Control Conference (CCC), 2018, pp. 10014–10019, doi: https://doi.org/10.23919/ChiCC.2018.8482830
  39. Z. Li et al., “Fast vision‐based autonomous detection of moving cooperative target for unmanned aerial vehicle landing,” J. F. Robot., vol. 36, no. 1, pp. 34–48, Jan. 2019, doi: https://doi.org/10.1002/rob.21815
  40. B. F. Jeon, H. J. Kim, “Online Trajectory Generation of a MAV for Chasing a Moving Target in 3D Dense Environments,” in 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2019, pp. 1115–1121, doi: https://doi.org/10.1109/IROS40897.2019.8967840
  41. B. F. Jeon, Y. Lee, J. Choi, J. Park, H. J. Kim, “Autonomous Aerial Dual-Target Following Among Obstacles,” IEEE Access, vol. 9, pp. 143104–143120, 2021, doi: https://doi.org/10.1109/ACCESS.2021.3117314
  42. L. Y. Lo, C. H. Yiu, Y. Tang, A.-S. Yang, B. Li, C.-Y. Wen, “Dynamic Object Tracking on Autonomous UAV System for Surveillance Applications,” Sensors, vol. 21, no. 23, p. 7888, Nov. 2021, doi: https://doi.org/10.3390/s21237888
  43. T. Naseer, J. Sturm, D. Cremers, “FollowMe: Person following and gesture recognition with a quadrocopter,” in 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, Nov. 2013, pp. 624–630, doi: https://doi.org/10.1109/IROS.2013.6696416
  44. F. Mueller, E. Graether, C. Toprak, “Joggobot,” in CHI ’13 Extended Abstracts on Human Factors in Computing Systems, Apr. 2013, pp. 2845–2846, doi: https://doi.org/10.1145/2468356.2479541
  45. R. Jin, J. Jiang, Y. Qi, D. Lin, T. Song, “Drone Detection and Pose Estimation Using Relational Graph Networks,” Sensors, vol. 19, no. 6, p. 1479, Mar. 2019, doi: https://doi.org/10.3390/s19061479
  46. T. Xiang et al., “UAV based target tracking and recognition,” in 2016 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI), Sep. 2016, pp. 400–405, doi: https://doi.org/10.1109/MFI.2016.7849521
  47. B. Penin, P. R. Giordano, and F. Chaumette, “Vision-Based Reactive Planning for Aggressive Target Tracking While Avoiding Collisions and Occlusions,” IEEE Robot. Autom. Lett., vol. 3, no. 4, pp. 3725–3732, Oct. 2018, doi: https://doi.org/10.1109/LRA.2018.2856526
  48. P. M. Wyder et al., “Autonomous drone hunter operating by deep learning and all-onboard computations in GPS-denied environments,” PLoS One, vol. 14, no. 11, p. e0225092, Nov. 2019, doi: https://doi.org/10.1371/journal.pone.0225092
  49. J. Thomas, G. Loianno, J. Polin, K. Sreenath, and V. Kumar, “Toward autonomous avian-inspired grasping for micro aerial vehicles,” Bioinspir. Biomim., vol. 9, no. 2, p. 025010, May 2014, doi: https://doi.org/10.1088/1748-3182/9/2/025010
  50. J. Thomas, G. Loianno, K. Sreenath, V. Kumar, “Toward image based visual servoing for aerial grasping and perching,” in 2014 IEEE International Conference on Robotics and Automation (ICRA), May 2014, pp. 2113–2118, doi: https://doi.org/10.1109/ICRA.2014.6907149
  51. H. Seo, S. Kim, H. J. Kim, “Aerial grasping of cylindrical object using visual servoing based on stochastic model predictive control,” in 2017 IEEE International Conference on Robotics and Automation (ICRA), May 2017, pp. 6362–6368, doi: https://doi.org/10.1109/ICRA.2017.7989751
  52. P. E. Pounds and A. M. Dollar, “Aerial Grasping from a Helicopter UAV Platform,” Experimental Robotics. 2014, pp. 269–283.
  53. L. Li et al., “Autonomous Removing Foreign Objects for Power Transmission Line by Using a Vision-Guided Unmanned Aerial Manipulator,” J. Intell. Robot. Syst., vol. 103, no. 2, p. 23, 2021, doi: https://doi.org/10.1007/s10846-021-01482-3
  54. L. Lin, Y. Yang, H. Cheng, X. Chen, “Autonomous Vision-Based Aerial Grasping for Rotorcraft Unmanned Aerial Vehicles,” Sensors, vol. 19, no. 15, p. 3410, Aug. 2019, doi: https://doi.org/10.3390/s19153410
  55. K. Su and S. Shen, “Catching a Flying Ball with a Vision-Based Quadrotor,” in 2016 International Symposium on Experimental Robotics, 2017, pp. 550–562, doi: https://doi.org/10.1007/978-3-319-50115-4_48
  56. K.-H. Zeng, R. Mottaghi, L. Weihs, and A. Farhadi, “Visual Reaction: Learning to Play Catch With Your Drone,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 11570–11579, doi: https://doi.org/10.1109/CVPR42600.2020.01159
  57. M. Vrba et al., “Autonomous capture of agile flying objects using UAVs: The MBZIRC 2020 challenge,” Rob. Auton. Syst., vol. 149, p. 103970, 2022, doi: https://doi.org/10.1016/j.robot.2021.103970
  58. M. Garcia, R. Caballero, F. Gonzalez, A. Viguria, and A. Ollero, “Autonomous drone with ability to track and capture an aerial target,” in 2020 International Conference on Unmanned Aircraft Systems (ICUAS), 2020, pp. 32–40, doi: https://doi.org/10.1109/ICUAS48674.2020.9213883
  59. J. Thomas, J. Welde, G. Loianno, K. Daniilidis, and V. Kumar, “Autonomous Flight for Detection, Localization, and Tracking of Moving Targets With a Small Quadrotor,” IEEE Robot. Autom. Lett., vol. 2, no. 3, pp. 1762–1769, 2017, doi: https://doi.org/10.1109/LRA.2017.2702198
  60. P. Ramon-Soria, B. C. Arrue, A. Ollero, “Grasp Planning and Visual Servoing for an Outdoors Aerial Dual Manipulator,” Engineering, vol. 6, no. 1, pp. 77–88, 2020, doi: https://doi.org/10.1016/j.eng.2019.11.003
  61. G. Zhang et al., “Grasp a Moving Target from the Air: System & Control of an Aerial Manipulator,” in 2018 IEEE International Conference on Robotics and Automation (ICRA), 2018, pp. 1681–1687, doi: https://doi.org/10.1109/ICRA.2018.8461103
  62. L. A. Tony et al., “Collaborative Tracking and Capture of Aerial Object using UAVs,” ArXiv, vol. abs/2010.01588, 2020, [Online]. Available: https://api.semanticscholar.org/CorpusID:222133245
  63. R. Ritz, M. W. Müller, M. Hehn, R. D’Andrea, “Cooperative quadrocopter ball throwing and catching,” in 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, Oct. 2012, pp. 4972–4978, doi: https://doi.org/10.1109/IROS.2012.6385963
  64. R. Opromolla, G. Fasano, and D. Accardo, “A Vision-Based Approach to UAV Detection and Tracking in Cooperative Applications,” Sensors, vol. 18, no. 10, p. 3391, 2018, doi: https://doi.org/10.3390/s18103391
  65. H. Lee, H. Kim, W. Kim, H. J. Kim, “An Integrated Framework for Cooperative Aerial Manipulators in Unknown Environments,” IEEE Robot. Autom. Lett., vol. 3, no. 3, pp. 2307–2314, 2018, doi: https://doi.org/10.1109/LRA.2018.2807486
  66. Y. Becerra, “Una revisión de plataformas robóticas para el sector de la construcción,” Tecnura, vol. 24, no. 63, pp. 115–132, 2020, doi: https://doi.org/10.14483/22487638.15384