Skip to main navigation menu Skip to main content Skip to site footer
International Journal of Robotics and Automation Technology

Articles

Vol. 2 No. 1 (2015)

Particle Filter-Based Robust Visual Servoing for UCF-MANUS-An Intelligent Assistive Robotic Manipulator

DOI
https://doi.org/10.31875/2409-9694.2015.02.01.4
Submitted
May 10, 2015
Published
10.05.2015

Abstract

A particle filter based tracking scheme is proposed to robustify visual servoing of objects in the UCF-MANUS camera-in-hand vision setup. Instead of simply fusing global and local information, a concatenation of the two sources of information is proposed here which enables the combination of the two independent measurements with a synergistic collaboration between them. A novel overlap metric to encode the degree and quality of overlap between two arbitrarily shaped Regions of Interest (ROIs) is defined to facilitate the prior and posterior pdfs in the particle filter setup.A sub-ROI is defined and utilized in the observation step to facilitate the global target detection. Based on extensive experimental results under a variety of scenarios obtained by using the UCF-MANUS assistive robotic testbed, it is seen that the proposed particle filter based fusion approach is superior to other non-fused global detection or local tracking approaches. The efficacy of the proposed approach has also been verified using standard data sets. Finally, robustification of a hybrid visual servoing technique is shown by implementing the proposed particle-filter based tracker during closed-loop operation in real-time.

References

  1. Ishiguro H, Yamamoto M and Tsuji S. Omni-directional Stereo for Making Global Map. in Proc IEEE International Conference on Computer Vision Osaka, Japan 1990; 540- 547. http://dx.doi.org/10.1109/iccv.1990.139591
  2. Roumeliotis SI and Bekey GA. An Extended Kalman Filter for frequent local and infrequent global sensor data fusion, in Proc. SPIE Conference on Sensor Fusion and Decentralized Control in Autonomous Robotic Systems, Pittsburgh, PA 1999; 11-22.
  3. Fu Y, Xu H, Li H, Wang S and Xu H. A Navigation Strategy based on Global Geographical Planning and Local Feature Positioning for Mobile Robot in Large Unknown Environment, in Proc. International Symposium on Systems and Control in Aerospace and Astronautics, Harbin, China 2006; 1189- 1193.
  4. Lee JH and Jung S. Global Position Tracking Control of an Omni-directional Mobile Robot Using Fusion of a Magnetic Compass and Encoders, in Proc. IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems, Seoul, Korea 2008; 246-251. http://dx.doi.org/10.1109/mfi.2008.4648072
  5. Moore DC, Huang AS, Walter M, Olson E, Fletcher L, Leonard J and Teller S. Simultaneous Local and Global State Estimation for Robotic Navigation, in Proc. IEEE International Conference on Robotics and Automation, pp. 3794- 3799,Kobe, Japan, 2009. http://dx.doi.org/10.1109/robot.2009.5152763
  6. Se S, Lowe DG and Little JJ. Vision-Based Global Localization and Mapping for Mobile Robots, IEEE Trans. On Robotics 2005; 21(3): 364-375. http://dx.doi.org/10.1109/TRO.2004.839228
  7. Lowe D. Distinctive Image Features from Scale-invariant Keypoints, International Journal of Computer Vision 2004; 60(2): 91-110. http://dx.doi.org/10.1023/B:VISI.0000029664.99615.94
  8. Rodriguez-Losada D, Matia F, Jimenez A and Galan R. Local Map Fusion for Real-time Indoor Simultaneous Localization and Mapping, Journal of Field Robotics 2006; 23(5): 291- 309. http://dx.doi.org/10.1002/rob.20120
  9. Persson M, Duckett T and Lilienthal AJ. Fusion of aerial images and sensor data from a ground vehicle for improved semantic mapping. Robotics and Autonomous Systems 2008; 56: 483-492. http://dx.doi.org/10.1016/j.robot.2008.03.002
  10. Pinheiro P and Lima P. Bayesian Sensor Fusion for Cooperative Object Localization and World Modeling, in Proc. the 8th Conference on Intelligent Autonomous Systems, Amsterdam, Netherlands, 2004.
  11. Ferrein A, Hermanns L and Lakemeyer G. Comparing Sensor Fusion Techniques for Ball Position Estimation. RoboCup 2005, Lecture Notes in Computer Science 2005; 154-165.
  12. Kotecha, Jayesh H, Djuric PM. Gaussian particle filtering. Signal Processing, IEEE Transactions on 2003; 51(10): 2592-2601. http://dx.doi.org/10.1109/TSP.2003.816758
  13. Aydogmus O, Talu MF. Comparison of Extended-Kalmanand Particle-Filter-Based Sensorless Speed Control. Instrumentation and Measurement, IEEE Transactions on 2012; 61(2): 402, 410.
  14. Cappe O, Godsill SJ, Moulines E. An Overview of Existing Methods and Recent Advances in Sequential Monte Carlo, Proceedings of the IEEE 2007; 95(5) 899-924. http://dx.doi.org/10.1109/JPROC.2007.893250
  15. Wang J and Yagi Y. Adaptive Mean-Shift Tracking With Auxiliary Particles, IEEE Trans. on Systems, Man, and Cybernetics---Part B: Cybernetics 2009; 39(6): 1578-1589. http://dx.doi.org/10.1109/TSMCB.2009.2021482
  16. Okuma K, Taleghani A, Freitas D, Little JJ and Lowe DG. A Boosted Particle Filter: Multitarget Detection and Tracking, in Proc. European Conf. Computer Vision, 2004. http://dx.doi.org/10.1007/978-3-540-24670-1_3
  17. Ai YLH, Yamashita T, Lao S and Kawade M. Tracking in Low Frame Rate Video: A Cascade Particle Filter with Discriminative Observers of Different Life Spans, IEEE Trans. on Pattern Analysis and Machine Intelligence 2008; 30(10): 1728-1740. http://dx.doi.org/10.1109/TPAMI.2008.73
  18. Chen H and Li Y. Dynamic View Planning by Effective Particles for Three-Dimensional Tracking, IEEE Trans. On Systems, Man, and Cybernetics---Part B: Cybernetics 2009; 39(1): 242-253. http://dx.doi.org/10.1109/TSMCB.2008.2005113
  19. Breitenstein M, Reichlin F, Leibe B, Koller-Meier E and Gool L. Robust Tracking-by-Detection using a Detector Confidence Particle Filter, In Proc. IEEE International Conference on Computer Vision 2009; 1515-1522. Kyoto, Japan, 2009. http://dx.doi.org/10.1109/iccv.2009.5459278
  20. Breitenstein M, Reichlin F, Leibe B, Koller-Meier E and Gool L. Online Multi-Person Tracking-by-Detection from a Single, Uncalibrated Camera, IEEE Trans. on Pattern Analysis and Machine Intelligence, accepted, to be publised.
  21. Huang C and L Fu. Multitarget Visual Tracking Based Effective Surveillance With Cooperation of Multiple Active Cameras, IEEE Trans. on Systems, Man, and Cybernetics--- Part B: Cybernetics 2011; 41(1): 234-247. http://dx.doi.org/10.1109/TSMCB.2010.2050878
  22. Wang Q, Chen F and Xu W. Tracking by Third-Order Tensor Representation, IEEE Trans. on Systems, Man, and Cybernetics---Part B: Cybernetics 2011; 41(2): 385-396. http://dx.doi.org/10.1109/TSMCB.2010.2056366
  23. Peurum P, Venkatesh S and West G. A study on smoothing for particle-filtered 3-D human body tracking. Int J Comput Vision 2010; 87(1-2): 53-74. http://dx.doi.org/10.1007/s11263-009-0205-5
  24. Won SP, Melek WW and Golnaraghi F. A Kalman/Particle Filter-Based Position and Orientation Estiamtion Method Using a Position Sensor/Inertial Measurement Unit Hybrid System, IEEE Trans Industrial Electronics 2010; 57(5): 1787- 1798. http://dx.doi.org/10.1109/TIE.2009.2032431
  25. Kristan M, Kovacic S, Leonardis A and Pers J. A Two-Stage Dynamic Model for Visual Tracking, IEEE Trans. on Systems, Man, and Cybernetics---Part B: Cybernetics 2010; 40(6): 1505-1520. http://dx.doi.org/10.1109/TSMCB.2010.2041662
  26. del Rincon JM, Makris D, Urunuela CO and Nebel JC. Tracking Human Position and Lower Body Parts Using Kalman and Particle Filters Constrained by Human Biomechanics, IEEE Trans. on Systems Man and Cybernetics---Part B: Cybernetics 2011; 41(1): 26-37. http://dx.doi.org/10.1109/TSMCB.2010.2044041
  27. Widynski N, Dubuisson S, Bloch I. Integration of Fuzzy Spatial Information in Tracking Based on Particle Filtering, IEEE Trans. on Systems, Man, and Cybernetics---Part B: Cybernetics 2011; 41(3): 635-649. http://dx.doi.org/10.1109/TSMCB.2010.2064767
  28. Kim DJ, Wang Z and Behal A. Motion Segmentation and Control Design for UCF-MANUS - An Intelligent Assistive Robotic Manipulator, IEEE/ASME Transactions on Mechatronics 2012; 17(5): 936-948. http://dx.doi.org/10.1109/TMECH.2011.2149730
  29. Kim DJ, Wang Z, Paperno N and Behal A. System Design and Implementation of UCF- MANUS - An Intelligent Assistive Robotic Manipulator, IEEE/ASME Trans. On Mechatronics 2014; 19(1): 225-237. http://dx.doi.org/10.1109/TMECH.2012.2226597
  30. Hutchinson S, Hager G and Corke P. A Tutorial on Visual Servo Control, IEEE Trans. Robot. Automat 1997; 13: 582- 595.
  31. Grabner H, Grabner M and Bischof H. Real-Time Tracking via Online Boosting, in Proc. Conf. British Machine Vision 2006; 47-56.
  32. Ozuysal M, Fua P and Lepetit V. Fast Keypoint Recognition in Ten Lines of Code, in Proc. IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, Minnesota 2007; 1-8. http://dx.doi.org/10.1109/cvpr.2007.383123
  33. Shi J and Tomasi C. Good Features to Track, in Proc. IEEE Conference on Computer Vision and Pattern Recognition 1994; 593-600.
  34. Birchfield S. Source Code for the KLT Feature Tracker, http://www.ces.clemson.edu/~stb/klt/, 2006.
  35. Silverman BW. Density Estimation for Statistics and Data Analysis, Vol. 26, CRC press 1986. http://dx.doi.org/10.1007/978-1-4899-3324-9
  36. Ross D, Lim J. Lin RS and Yang MH. Incremental Learning for Robust Visual Tracking, International Journal of Computer Vision 2008; 77(1): 125-141. http://dx.doi.org/10.1007/s11263-007-0075-7
  37. Bay H, Ess A, Tuytelaars T and Van Gool L. SURF: Speeded Up Robust Features, Computer Vision and Image Understanding (CVIU) 2008; 110(3): 346-359. http://dx.doi.org/10.1016/j.cviu.2007.09.014
  38. Fang Y, Behal A, Dixon WE and Dawson DM. Adaptive 2.5D visual servoing of kinematically redundant robot manipulators, Proc. of the 41 IEEE Conference on Decision and Control 2002; 3: 2860, 2865. 10-13. http://dx.doi.org/10.1109/cdc.2002.1184282
  39. Yilmaz A, Javed O and Shah M. Object tracking: A survey, ACM Comput Surv 2006; 38(4): 1-45. http://dx.doi.org/10.1145/1177352.1177355
  40. Arulampalam S, Maskell S, Gordon N and Clapp T. A tutorial on particle filter for on-line nonlinear/non-Gaussian Bayesian tracking, IEEE Trans. Signal Processing 2001; 50(2): 174-188. http://dx.doi.org/10.1109/78.97837441
  41. Rekleitis IM. A Particle Filter Tutorial for Mobile Robot Localization, Centre for Intelligent Machines, McGill University, Technical Report TR-CIM-04-02, 2004.
  42. Liu JS, Chen R and Logvinenko T. A theoretical framework for sequential importance sampling and resampling, In A. Doucet N. de Freitas, and NJ. Gordon, editors, Sequential Monte Carlo in Practice, Springer-Verlag 2001.
  43. Sinha S. Source Code for the GPU-KLT algorithm, http://www.cs.unc.edu/~ssinha/Research/GPU_KLT/.