Research Papers

Estimating Endoscopic Orientation in Static and Dynamic States With Inertial Sensors

[+] Author and Article Information
Hung V. Dao

Graduate School of Engineering and Science,
Shibaura Institute of Technology,
Saitama City,
Saitama 337-8570, Japan
e-mail: nb13503@shibaura-it.ac.jp

Takashi Komeda

Bioscience and Engineering,
Shibaura Institute of Technology,
Saitama City,
Saitama 337-8570, Japan
e-mail: komeda@se.shibaura-it.ac.jp

Manuscript received September 17, 2015; final manuscript received March 14, 2016; published online August 5, 2016. Assoc. Editor: Carl Nelson.

J. Med. Devices 10(4), 041003 (Aug 05, 2016) (7 pages) Paper No: MED-15-1261; doi: 10.1115/1.4033332 History: Received September 17, 2015; Revised March 14, 2016

This paper presents a new method for estimating the tilt angles of endoscopic images. Disorientation is one of the major challenges during natural orifice translumenal endoscopic surgery (NOTES). Reorientation allows surgeons or gastroenterologists to work in off-axis conditions and provides an important reference for coupling a secondary image. Some published studies of angle estimation for NOTES still have the limitation under the influence of movement or vibration. This study proposes a new sensor-fusion method for reducing the shock-based error. A triaxial accelerometer measures the gravitational vector (g-components) in all static states. When motion appears, the angular velocity from a triaxial gyroscope is used to calculate the elemental changes in g-components. A so-called predict-and-choose process relies on this data to predict the future state by giving many prediction values. The relationship between these values, the newest accelerometer readings, and their variation determine the final choice. Hence, under all conditions, the gravitational components are iteratively estimated to calculate the tilt angles. The result is evaluated by being applied in a well-known application, endoscopic horizon stabilization. Compared with the reference method, the proposed method has notable advantages. The simulation and experimental results show small errors, smooth angle change, and a small delay time. The tilt angles are reliable without any cumulative error under the prolonged motion. Therefore, this study gives surgeons or gastroenterologists an improved rectified image for reorienting under off-axis conditions. Further research will identify more applications for the development of surgical instruments for NOTES.

Copyright © 2016 by ASME
Your Session has timed out. Please sign back in to continue.


Dray, X. , and Kalloo, A. N. , 2012, “ History of NOTES,” Natural Orifice Translumenal Endoscopic Surgery: Textbook and Video Atlas, 1st ed., A. N. Kalloo , J. Marescaux , and R. Zorron , eds., Wiley-Blackwell, Hoboken, NJ, pp. 3–11.
McGee, M. F. , Rosen, M. J. , Marks, J. , Onders, R. P. , Chak, A. , Faulx, A. , Chen, V. K. , and Ponsky, J. , 2006, “ A Primer on Natural Orifice Transluminal Endoscopic Surgery: Building a New Paradigm,” Surg. Innovation, 13(2), pp. 86–93. [CrossRef]
Meadows, M. C. , and Chamberlain, R. S. , 2010, “ A Review on the Status of Natural Orifice Transluminal Endoscopic Surgery (NOTES) Cholecystectomy: Techniques and Challenges,” Open Access Surg., 2010(3), pp. 73–86.
Rattner, D. , and Kalloo, A. , 2006, “ ASGE/SAGES Working Group on Natural Orifice Translumenal Endoscopic Surgery,” Surg. Endosc., 20(2), pp. 329–333. [CrossRef] [PubMed]
Höller, K. , Penne, J. , Schneider, A. , Jahn, J. , Guttiérrez, J. , Wittenberg, T. , Feußner, H. , and Hornegger, J. , 2009, “ Endoscopic Orientation Correction,” Medical Image Computing and Computer-Assisted Intervention—MICCAI (Lecture Notes in Computer Science), Vol. 5761, Springer, Berlin, pp. 459–466.
Höller, K. , Penne, J. , Hornegger, J. , Schneider, A. , Gillen, S. , Feußner, H. , Jahn, J. , Guttiérrez, J. , and Wittenberg, T. , 2009, “ Clinical Evaluation of Endorientation: Gravity Related Rectification for Endoscopic Images,” 6th International Symposium on Image and Signal Processing and Analysis (ISPA 2009), Salzburg, Austria, Sept. 16–18, pp. 713–717.
Höller, K. , Penne, J. , Schneider, A. , Jahn, J. , Girgis, H. , Guttiérrez, J. , Wittenberg, T. , Feußner, H. , and Hornegger, J. , 2009, “ Suppression of Shock Based Errors With Gravity Related Endoscopic Image Rectification,” 5th Russian-Bavarian Conference on Bio-Medical Engineering (RBC'™09), Munich, Germany, July 1–4, pp. 43–47.
Warren, A. , Mountney, P. , Noonan, D. , and Yang, G.-Z. , 2012, “ Horizon Stabilized—Dynamic View Expansion for Robotic Assisted Surgery (HS-DVE),” Int. J. Comput. Assisted Radiol. Surg., 7(2), pp. 281–288. [CrossRef]
Luinge, H. J. , and Veltink, P. H. , 2004, “ Inclination Measurement of Human Movement Using a 3-D Accelerometer With Autocalibration,” IEEE Trans. Neural Syst. Rehabil. Eng., 12(1), pp. 112–121. [CrossRef] [PubMed]
Luinge, H. J. , and Veltink, P. H. , 2005, “ Measuring Orientation of Human Body Segments Using Miniature Gyroscopes and Accelerometers,” Med. Biol. Eng. Comput., 43(2), pp. 273–282. [CrossRef] [PubMed]
Dejnabadi, H. , Jolles, B. M. , and Aminian, K. , 2005, “ A New Approach to Accurate Measurement of Uniaxial Joint Angles Based on a Combination of Accelerometers and Gyroscopes,” IEEE Trans. Biomed. Eng., 52(8), pp. 1478–1484. [CrossRef] [PubMed]
Williamson, R. , and Andrews, B. J. , 2001, “ Detecting Absolute Human Knee Angle and Angular Velocity Using Accelerometers and Rate Gyroscopes,” Med. Biol. Eng. Comput., 39(3), pp. 294–302. [CrossRef] [PubMed]
Luinge, H. J. , Veltink, P. H. , and Baten, C. T. M. , 1999, “ Estimating Orientation With Gyroscopes and Accelerometers,” Technol. Health Care, 7(6), pp. 455–459. [PubMed]
Duke, E. L. , Antoniewicz, R. F. , and Krambeer, K. D. , 1988, Derivation and Definition of a Linear Aircraft Model (NASA Reference Publication), Vol. 1207, National Aeronautics and Space Administration (NASA), Edwards, CA.
Pedley, M. , 2013, “ Tilt Sensing Using a Three-Axis Accelerometer,” Freescale Semiconductor, Inc., Austin, TX, Application Note No. AN3461 Rev. 6.
Diebel, J. , 2006, “ Representing Attitude: Euler Angles, Unit Quaternions, and Rotation Vectors,” Stanford University, Stanford, CA.
Looney, M. , 2010, “ A Simple Calibration for MEMS Gyroscopes,” EDN Europe.
Pedley, M. , 2013, “ High Precision Calibration of a Three-Axis Accelerometer,” Freescale Semiconductor, Inc., Austin, TX, Application Note No. AN4399 Rev. 1.
Sodergren, M. H. , Warren, A. , Nehme, J. , Clark, J. , Gillen, S. , Feussner, H. , Teare, J. , Darzi, A. , and Yang, G.-Z. , 2014, “ Endoscopic Horizon Stabilization in Natural Orifice Translumenal Endoscopic Surgery: A Randomized Controlled Trial,” Surg. Innovation, 21(1), pp. 74–79. [CrossRef]
Rattner, D. W. , Hawes, R. , Schwaitzberg, S. , Kochman, M. , and Swanstrom, L. , 2011, “ The Second SAGES/ASGE White Paper on Natural Orifice Transluminal Endoscopic Surgery: 5 Years of Progress,” Surg. Endosc., 25(8), pp. 2441–2448. [CrossRef] [PubMed]
Lehman, A. C. , Dumpert, J. , Wood, N. A. , Redden, L. , Visty, A. Q. , Farritor, S. , Varnell, B. , and Oleynikov, D. , 2009, “ Natural Orifice Cholecystectomy Using a Miniature Robot,” Surg. Endosc., 23(2), pp. 260–266. [CrossRef] [PubMed]
Quaglia, C. , Petroni, G. , Niccolini, M. , Caccavaro, S. , Dario, P. , and Menciassi, A. , 2014, “ Design of a Compact Robotic Manipulator for Single-Port Laparoscopy,” ASME J. Mech. Des., 136(10), p. 105001. [CrossRef]


Grahic Jump Location
Fig. 4

Gravitational vector changes caused by each elemental rotation: (a) rotation about Z-axis changes gx and gy, (b) rotation about the X-axis changes gy and gz, and rotation about the Y-axis changes gz and gx

Grahic Jump Location
Fig. 3

Sensor fusion in the predict-and-choose process

Grahic Jump Location
Fig. 2

Tip of the flexible endoscope: (a) position of the IMU and (b) definition of the three elemental rotations

Grahic Jump Location
Fig. 1

System overview with key components and key processes in the basic application

Grahic Jump Location
Fig. 8

Simulation results under condition (iv), combined linear acceleration and rotation: akmax=3 m/s2 and ω≠0

Grahic Jump Location
Fig. 9

Simulation results under condition (iv), combined linear acceleration and rotation: akmax=10 m/s2 and ω≠0

Grahic Jump Location
Fig. 5

Generating the simulation data by the virtual IMU: (a) schematic diagram of the virtual IMU and (b) original tilt angles for conditions (iii) and (iv); here, Φ changes as a sine function while Θ changes as a linear function

Grahic Jump Location
Fig. 6

Simulation results under condition (ii), linear acceleration only: akmax=3 m/s2 and ω = 0

Grahic Jump Location
Fig. 7

Simulation results under condition (iii), rotation with small background vibration: akmax=0.5 m/s2 and ω≠0

Grahic Jump Location
Fig. 10

Rotation frame used in the experiments

Grahic Jump Location
Fig. 11

Experimental results when the motion noise is small

Grahic Jump Location
Fig. 12

Experimental results under the strong motion

Grahic Jump Location
Fig. 13

Using the output angles of both methods to stabilize the horizon under some conditions: (a) static state, (b) weak motion, (c) strong motion, and (d) continuous strong motion



Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In