Research Papers

Robust Automatic Feature Tracking on Beating Human Hearts for Minimally Invasive CABG Surgery

[+] Author and Article Information
H. Mohamadipanah

Department of Surgery,
University of Wisconsin,
Madison, WI 53792
e-mail: hmohamadipan@wisc.edu

M. Andalibi

Department of Mechanical Engineering,
Embry-Riddle Aeronautical University,
Prescott, AZ 86301
e-mail: Mehran.Andalibi@erau.edu

L. Hoberock

Fellow ASME
Department of Mechanical Engineering,
Oklahoma State University,
Stillwater, OK 74075
e-mail: larry.hoberock@okstate.edu

Manuscript received September 2, 2015; final manuscript received April 1, 2016; published online September 14, 2016. Assoc. Editor: Carl Nelson.

J. Med. Devices 10(4), 041010 (Sep 14, 2016) (8 pages) Paper No: MED-15-1251; doi: 10.1115/1.4033301 History: Received September 02, 2015; Revised April 01, 2016

This paper presents a robust algorithm for automatic tracking of feature points on the human heart. The emphases and key contributions of the proposed algorithm are uniform distribution of the feature points and sustained tolerable tracking error. While in many methods in the literature, detection takes place independently from the tracking procedure, adopting a different approach, we selected a data-driven detection stage, which works based on the feedback from tracking results from the Lucas–Kanade (LK) tracking algorithm to avoid unacceptable tracking errors. To ensure a uniform spatial distribution of the total detected feature points for tracking, a cost function is employed using the simulated annealing optimizer, which prevents the newly detected points from accumulating near the previously located points or stagnant regions. Implementing the proposed algorithm on a real human heart dataset showed that the presented algorithm yields more robust tracking and improved motion reconstruction, compared with the other available methods. Furthermore, to predict the motion of feature points for handling short-term occlusions, a state space model is utilized, and thin-plate spline (TPS) interpolation was also employed to estimate motion of any arbitrary point on the heart surface.

Copyright © 2016 by ASME
Your Session has timed out. Please sign back in to continue.


Freschi, C. , Ferrari, V. , Melfi, F. , Ferrari, M. , Mosca, F. , and Cuschieri, A. , 2013, “ Technical Review of the da Vinci Surgical Telemanipulator,” Int. J. Med. Rob. Comput. Assisted Surg., 9(4), pp. 396–406. [CrossRef]
Poston, R. S. , Tran, R. , Collins, M. , Reynolds, M. , Connerney, I. , Reicher, B. , Zimrin, D. , Griffith, B. P. , and Bartlett, S. T. , 2008, “ Comparison of Economic and Patient Outcomes With Minimally Invasive Versus Traditional Off-Pump Coronary Artery Bypass Grafting Techniques,” Ann. Surg., 248(4), pp. 638–646. [PubMed]
Inderbitzi, R. G. C. , Schmid, R. A. , Melfi, F. M. A. , and Casula, R. P. , 2012, Minimally Invasive Thoracic and Cardiac Surgery, Springer, Berlin.
Lemma, M. , Mangini, A. , Redaelli, A. , and Acocella, F. , 2005, “ Do Cardiac Stabilizers Really Stabilize? Experimental Quantitative Analysis of Mechanical Stabilization,” Interact. Cardiovasc. Thorac. Surg., 4(3), pp. 222–226. [CrossRef] [PubMed]
Moustris, G. P. , Mantelos, A. I. , and Tzafestas, C. S. , 2013, “ Enhancing Surgical Accuracy Using Virtual Fixtures and Motion Compensation in Robotic Beating Heart Surgery,” 21st Mediterranean Conference on Control & Automation (MED), Platanias-Chania, Crete, Greece, June 25–28, pp. 1254–1260.
Nakamura, Y. , Kishi, K. , and Kawakami, H. , 2001, “ Heartbeat Synchronization for Robotic Cardiac Surgery,” IEEE International Conference on Robotics and Automation (ICRA), Seoul, South Korea, May 21–26, Vol. 2, pp. 2014–2019.
Bebek, O. , and Cavusoglu, M. , 2007, “ Intelligent Control Algorithms for Robotic-Assisted Beating Heart Surgery,” IEEE Trans. Rob., 23(3), pp. 468–480. [CrossRef]
Ginhoux, R. , Gangloff, J. , Mathelin, M. D. , Soler, L. , Sanchez, M. , and Marescaux, J. , 2005, “ Active Filtering of Physiological Motion in Robotized Surgery Using Predictive Control,” IEEE Trans. Rob., 21(1), pp. 67–79. [CrossRef]
Groeger, M. , Ortmaier, T. , Sepp, W. , and Hirzinger, G. , 2002, “ Tracking Local Motion on the Beating Heart,” Proc. SPIE, 4681, pp. 233–241.
Noce, A. , Triboulet, J. , and Poignet, P. , 2007, “ Efficient Tracking of the Heart Using Texture,” 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBS), Lyon, France, Aug. 22–26, pp. 4480–4483.
Devernay, F. , Mourgues, F. , and Coste-Maniere, E. , 2001, “ Towards Endoscopic Augmented Reality for Robotically Assisted Minimally Invasive Cardiac Surgery,” International Workshop on Medical Imaging and Augmented Reality (MIAR 2001), Shatin, Hong Kong, June 10–12, pp. 16–20.
Tuna, E. , Franke, T. , Bebek, O. , Shiose, A. , Fukamachi, K. , and Cavusoglu, M. , 2013, “ Heart Motion Prediction Based on Adaptive Estimation Algorithms for Robotic-Assisted Beating Heart Surgery,” IEEE Trans. Rob., 29(1), pp. 261–276. [CrossRef]
Mountney, P. , Stoyanov, D. , and Yang, G. Z. , 2010, “ Three-Dimensional Tissue Deformation Recovery and Tracking,” IEEE Signal Process. Mag., 27(4), pp. 14–24. [CrossRef]
Sauvee, M. , Noce, A. , Poignet, P. , Triboulet, J. , and Dombre, E. , 2007, “ Three-Dimensional Heart Motion Estimation Using Endoscopic Monocular Vision System: From Artificial Landmarks to Texture Analysis,” Biomed. Signal Process. Control, 2(3), pp. 199–207. [CrossRef]
Richa, R. , Bo, A. P. , and Poignet, P. , 2011, “ Towards Robust 3D Visual Tracking for Motion Compensation in Beating Heart Surgery,” Med. Image Anal., 15(3), pp. 302–315. [CrossRef] [PubMed]
Elhawary, H. , and Popovic, A. , 2011, “ Robust Feature Tracking on the Beating Heart for a Robotic-Guided Endoscope,” Int. J. Med. Rob. Comput. Assisted Surg., 7(4), pp. 459–468. [CrossRef]
Harris, C. , and Stephens, M. , 1988, “ A Combined Corner and Edge Detector,” 4th Alvey Vision Conference (AVC88), Manchester, UK, Aug. 31–Sept. 2, pp. 147–151.
Bay, H. , Ess, A. , Tuytelaars, T. , and Gool, L. V. , 2008, “ Speeded-Up Robust Features (SURF),” Comput. Vision Image Understanding, 110(3), pp. 346–359. [CrossRef]
Stoyanov, D. , Mylonas, G. P. , Deligianni, F. , Darzi, A. , and Yang, G. Z. , 2005, “ Soft-Tissue Motion Tracking and Structure Estimation for Robotic Assisted MIS Procedures,” 8th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI’05), Palm Springs, CA, Oct. 26–29, pp. 139–146.
Humphrey, J. , 2010, Cardiovascular Solid Mechanics: Cells, Tissues, and Organs, Springer, New York.
Lucas, B. D. , and Kanade, T. , 1981, “ An Iterative Image Registration Technique With an Application to Stereo Vision,” 7th International Joint Conference on Artificial Intelligence (IJCAI’81), Vancouver, BC, Canada, Aug. 24–28, Morgan Kaufmann Publishers, San Francisco, CA, Vol. 2, pp. 674–679.
Mohamadipanah, H. , Hoberock, L. , and Andalibi, M. , 2015, “ Predictive Model Reference Adaptive Controller to Compensate Heart Motion in Minimally Invasive CABG Surgery,” Cardiovasc. Eng. Technol., 6(3), pp. 329–339.
Giannarou, S. , Stoyanov, D. , Noonan, D. , Mylonas, G. , Clark, J. , Visentini-Scarzanella, M. , Mountney, P. , and Yang, G.-Z. , 2015, “Hamlyn Centre Laparoscopic/Endoscopic Video Datasets,” Imperial College London, London, UK, accessed Feb. 23, 2016, http://hamlyn.doc.ic.ac.uk/vision/
Montgomery, H. L. H. L. , 1994, Ten Lectures on the Interface Between Analytic Number Theory and Harmonic Analysis (Regional Conference Series in Mathematics), American Mathematical Society, Providence, RI.
Kullback, S. , and Leibler, R. A. , 1951, “ On Information and Sufficiency,” Ann. Math. Stat., 22(1), pp. 79–86. [CrossRef]
Kirkpatrick, S. , Gelatt, C. D. , and Vecchi, M. P. , 1983, “ Optimization by Simulated Annealing,” Science, 220(4598), pp. 671–680. [CrossRef] [PubMed]
Bogatyrenko, E. , Pompey, P. , and Hanebeck, U. , 2011, “ Efficient Physics-Based Tracking of Heart Surface Motion for Beating Heart Surgery Robotic Systems,” Int. J. Comput. Assisted Radiol. Surg., 6(3), pp. 387–399. [CrossRef]
Ljung, L. , ed., 1999, System Identification: Theory for the User, 2nd ed., Prentice Hall, Upper Saddle River, NJ.
Marquardt, D. W. , 1963, “ An Algorithm for Least-Squares Estimation of Nonlinear Parameters,” SIAM J. Appl. Math., 11(2), pp. 431–441. [CrossRef]
Sun, C. , and Hahn, J. , 2006, “ Parameter Reduction for Stable Dynamical Systems Based on Hankel Singular Values and Sensitivity Analysis,” Chem. Eng. Sci., 61(16), pp. 5393–5403. [CrossRef]
Liu, W. , and Ribeiro, E. , 2011, “ A Survey on Image-Based Continuum-Body Motion Estimation,” Image Vision Comput., 29(8), pp. 509–523. [CrossRef]
Bookstein, F. L. , 1989, “ Principal Warps: Thin-Plate Splines and the Decomposition of Deformations,” IEEE Trans. Pattern Anal. Mach. Intell., 11(6), pp. 567–585. [CrossRef]
Kroon, D. J. , “ Lucas Kanade Affine Template Tracking,” MathWorks Inc., Natick, MA, accessed Feb. 23, 2016, http://www.mathworks.com/matlabcentral
Dodge, J. , Brown, B. G. , Bolson, E. L. , and Dodge, H. T. , 1992, “ Lumen Diameter of Normal Human Coronary Arteries. Influence of Age, Sex, Anatomic Variation, and Left Ventricular Hypertrophy or Dilation,” Circulation, 86(1), pp. 232–246. [CrossRef] [PubMed]


Grahic Jump Location
Fig. 3

Minimum individual distances from grid points to two different collections of feature points: sum of minimum distances to a spatially uniform collection of feature points is smaller

Grahic Jump Location
Fig. 2

Flowchart for the proposed algorithm

Grahic Jump Location
Fig. 1

Detected feature points on a beating human heart using our proposed method, the Harris corner detector, and SURF in an ROI. Result of tracking point A demonstrated in Figs. 6 and 7. The dataset, video of a CABG procedure on real human heart tissue, was taken from Ref. [19].

Grahic Jump Location
Fig. 5

Hankel singular values of the state space model versus different model orders

Grahic Jump Location
Fig. 6

Handling short-term occlusion for the X component of feature point A in Fig. 1 using state space model prediction

Grahic Jump Location
Fig. 7

Handling short-term occlusion for the Y component of feature point A in Fig. 1 using state space model prediction

Grahic Jump Location
Fig. 4

(a) Detection initialization in the four corners of ROI, (b) and (c) intermediate steps of feature point detection, and (d) finally detected feature points; in each step, new detected feature points are shown by asterisks and detected feature points from the previous step are shown by circle

Grahic Jump Location
Fig. 8

Tracking a mesh of arbitrary points in consecutive frames: (a) frame 1, (b) frame 3, (c) frame 5, and (d) frame 7




Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In