[From Proceedings of the Human Factors Society 41st Annual Meeting, September 22-26, 1997. Copyright by the Human Factors and Ergonomics Society, P.O. Box 1369, Santa Monica, CA 90406- 1369 USA; 310/394-1811, fax 310/394-2410, http://hfes.org. The complete article, including tables and figures, is available on request from HFES.]
Computer Assisted 3D Measurements for Micro-Surgery
Miriam Kim, Paul Milgram, James M. Drake, University of Toronto, Ontario, Canada
Abstract
A unique measuring tool, the Virtual Tape Measure (VTM), was developed in prototype form to minimize the inaccuracy surgeons encounter when trying to estimate the measurement of structures seen under the operating microscope. Superimposed on a stereo- video image of the surgical field taken through an operating microscope, the VTM is a three-dimensional measuring tool comprising a stereo-computer graphic pointer and a tethering line. The human operator simply places the VTM pointer adjacent two points he wishes to measure on the stereo-video image by controlling a three degrees of freedom (x, y, z-axes) input device. Active liquid shutter glasses must be worn to visualize the pointer and video images in stereo. To test the accuracy of the system, seven subjects used the VTM to measure distances between two dot models placed under the operating microscope. The absolute distance between the two dots ranged from 4 mm to 27 mm in a field of view measuring 20 mm x 30 mm. The distance in depth between the two dots ranged from 0 mm to 19 mm. Overall bias was an overestimation of 0.3 mm with a 95% confidence interval of 0.1 mm to 0.5 mm. The overall precision, as defined by variance of the error was 0.4 mm with a 95% confidence interval of 0.3 to 0.6 mm. Testing shows the VTM prototype system to be an accurate and reliable tool for measuring small objects with size scales typically encountered under the operating microscope during surgical procedures.
I. Introduction
Minimally Invasive Surgery (MIS) is a term which refers to any surgical procedure in which the surgeon accesses the surgical site by means of small incisions, rather than through wide openings, and operates with the aid of specially designed manipulators and viewing devices, to achieve minimal tissue destruction. Since inception of the concept, MIS techniques have revolutionized a broad range of surgical procedures and have had a major impact on patient safety and health care costs. The modern operating microscope, which both magnifies and illuminates the surgical site, is one of the key instruments used in MIS, and its use continues to facilitate difficult surgical procedures and realize the goals of minimally invasive surgery.
One of the critical limitations of the operating microscope, however, is the surgeon's inability to accurately determine the size of structures seen in the surgical field. This problem exists despite the existence of stereoscopic views, due to the use of variable magnifications and to the absence of a consistent known fixed frame of reference. That is, although familiarity with the magnified surgical space does develop with frequent use, and although relative sizes and distances are readily perceived, absolute measurements can still not be made reliably. Moreover, certain critical structures are often too small or fragile to be measured directly with such physical measuring instruments as calibrated probes or rulers since they may rupture or otherwise be damaged when touched.
Although placing a simple two dimensional scale in the frontal- parallel (x-y) plane may facilitate distance measurements between objects within the same depth plane, the problem of measuring distances between structures in different depth planes (z-axis) remains. To overcome this difficulty, we have created a prototype virtual tape measure (VTM) for the operating microscope, by employing the Augmented Reality technique of combining stereo- video images with stereo-computer graphics. The VTM is intended to enable surgeons to make measurements of any structure in the surgical field, throughout the operation, thereby providing accurate information for optimal decision making.
II. System Description
The stereo-video image of the surgical field is obtained by combining an operating microscope head (Zeiss Inc.) with an add- on stereo-video apparatus (Bristlecone Inc.), which consists of two separate cameras within a single housing to capture optical images from both left and right eye paths. The left and right eye images are displayed on one color monitor using alternating fields, at a rate of 120 fields per second. The proper view is directed to the corresponding eye through the use of synchronized active liquid shutter glasses. The live video images can both be fed into the computer for real time application of the VTM and simultaneously recorded onto a regular video recorder for subsequent analysis (See Fig. 1).
To apply the VTM, the stereo-video image signal is sampled with the video board of a SGI (Silicon Graphics Inc.) Indy workstation. The VTM is a three-dimensional computer generated pointer which appears to hover within the stereo video image upon which it is superimposed. (See Fig. 1 Insert B.) The VTM software is a special purpose application of the ARGOS (Augmented Reality through Graphic Overlays on Stereo-video) display system created in our lab [Drascic 1993, Milgram 1994].
To use the VTM, the human operator employs a three degree-of- freedom control device to manipulate the 3D pointer. The pointer is first placed adjacent to an object feature of interest in the stereo-video image. When the pointer is placed adjacent to a second location of interest, the absolute distance between the two points is computed and displayed.
Two additional features of the software enable improved accuracy of pointer placement. First, a tethering line can be drawn from the first location while the second location is sought, providing an extra perceptual aid to facilitate manipulation of the pointer in depth [Ruffo 1992]. Second, a computer assisted feature grabber allows the user to place the pointer more accurately when selected. This option is based on machine vision detection of edges and correlated features and provides the user with a second opinion of the location of the point of interest. Although the feature grabber has been shown to increase the reliability of the VTM [Milgram 1997], the human operator may decide to ignore rather than accept this second opinion. One reason for doing so is that the feature grabber option works best when there are sufficient distinguishing features between points of interest and their surroundings.
III. Experimental Evaluation
In order for the VTM system to operate precisely, accurately and reliably, two key conditions must be satisfied. First of all, it is necessary for the equivalent stereo camera system (shown in Fig. 1 Insert A) to be properly calibrated. As explained elsewhere [Rastogi 1995], this depends on the ability to measure the equivalent camera separation, convergence angle, and focal lengths. Secondly, it is necessary for the human operator to be capable of aligning the 3D pointer with points of interest in the stereo video image at acceptable levels of accuracy and precision. Whereas experimental assessment of earlier implementations of the VTM have demonstrated very good performance on a macroscopic scale [Milgram 1997, Drascic 1991], no results have been reported in the literature on precision and accuracy of the VTM on a microscopic scale. The present experiment was designed to address that issue.
To test the performance of the measurement system, seven subjects with acceptable visual acuity and stereo-acuity (depth discrimination of at least 30 seconds of arc at 40 cm using the Randot(R) stereopsis test) used the VTM to measure distances between two small dots (black on white background) placed under the operating microscope. The subjects' task was to sequentially align the VTM pointer to the black dot targets and then apply the "feature grab" option to complete the measurement. The absolute distance between the two dots as calculated by the VTM was recorded in each case. Eight pairs of targets were used, and subjects made only one measurement on each of the 8 models. Using a magnification of 16 times, the absolute distances between the two points ranged from 4 mm to 27 mm, in a fixed field of view measuring 20 mm x 30 mm. In the depth dimension, the distance between the two dots in each case ranged from 0 mm to 19 mm, as limited by the apparent depth of field of the operating microscope.
IV. Results
The results of the performance tests are shown on Figures 2 and 3. Two aspects of performance were investigated and quantified: accuracy and precision.
Accuracy, or absence of bias, was defined as any tendency of the subjects to neither consistently to underestimate (negative measurement error) nor overestimate (positive measurement error) the distances between the two dots using the VTM. Accuracy was therefore assessed in terms of the computed mean of the measurement error. As indicated in Fig. 2, there was a slight overall overestimation bias of 0.3 mm. The 95% confidence interval for accuracy is 0.1 mm to 0.5 mm. Figure 3 shows accuracy as analyzed by depth.
Precision, or consistency, was rated in terms of the spread of measurement error and was the calculated variance of the error. The overall variance was 0.4 mm, with a 95% confidence interval of 0.3 mm to 0.6 mm.
V. Discussion
In general, stereoscopic operating microscopes allow for added depth perception through retinal disparity cues not provided by monoscopic display systems. In spite of this, however, the problem of measuring absolute distances and dimensions remains. When combined with stereo-computer graphics, on the other hand, a unique measuring tool, the virtual tape measure, has been created. Under the favorable measuring conditions of the experiment reported here, the overall performance of the VTM has been shown to have a 95% confidence interval for the precision, or variance, of 0.4 mm to 0.6 mm, with a positive bias of 0.3 mm. These results thus support the potential of the VTM prototype system as an accurate and reliable tool for measuring small objects with size scales typically encountered under the operating microscope during surgical procedures.
In terms of specific applications, one potentially powerful use of this tool is for intraoperative measurement of the size of the "neck" of an aneurysm, to help decide on the most appropriate size of the clip which is used to occlude the aneurysm. At present, the only means that the surgeon has available for estimating this dimension is the technique if placing a probing instrument of known size adjacent to the aneurysm. As mentioned above, this procedure has some danger associated with it, in case the probe inadvertently touches the aneurysm and causes it to rupture.
One of the important considerations for practical use of the VTM in minimally invasive surgery, of course, is whether results from initial evaluation experiments will translate into comparable performance figures under actual intraoperative conditions. In the experiment reported here, the measurement error quantified was well within acceptable limits for most surgical procedures.
It is important to note that the small errors observed here may have resulted from inherent limitations in the human operator's ability to perform the alignment task, since this task relies solely on retinal disparity cues for aligning the virtual pointer to real object features. Other powerful depth cues normally used during real object to real object alignment tasks, such as distant object occlusion, consistent light and shadow effect, and texture gradient [Rastogi 1995, Surdick 1994] are lacking in systems that combine real and virtual objects. Another possible source of error is optical distortions in the stereo video system, that have not been taken into account by the current version of the Virtual Tape Measure.
VI. Conclusion
The VTM is a unique measuring tool which transforms difficult intraoperative absolute distance judgment tasks into a more amenable relative judgment task. Our current prototype has been shown here to have clinically acceptable accuracy and reliability. Potential applications of the VTM include any surgical procedures during which precise measurements of objects seen with the operating microscope are required. Planned future evaluations of the VTM include of actual intraoperative measurements, first in a simulated environment and then, as appropriate, in vivo. With further adaptation, the VTM is also expected to become an effective tool in conjunction with other minimally invasive surgical viewing systems, such as the stereo- endoscope.
VII. References
Drascic, D. & Milgram, P., "Positioning accuracy of a virtual stereographic pointer in a real stereoscopic video world", Proc. SPIE Vol. 1457: Stereoscopic Displays and Applications II, San Jose, CA, Feb. 1991.
Drascic D., Grodski J.J., Milgram P., Ruffo K., Wong P., Zhai S., 'ARGOS: A Display System for Augmenting Reality'. ACM SIGGRAPH Technical Video Review, Volume 88(7): InterCHI'93 Conf on Human Factors in Computing Systems, (abstract appears in Proceedings of InterCHI'93, p. 521), Amsterdam, April 1993.
Milgram, P., Takemura, H., Utsumi, A., Kishino, F., "Augmented reality: A class of displays on the reality-virtuality continuum", Proc. SPIE Vol. 2351: Telemanipulator and Telepresence Technologies, Boston, Oct. 1994.
Milgram, P., Yin, S., Grodski, J.J., "An augmented reality based teleoperation interface for unstructured environments", Proc. ANS 7th Topical Meeting on Robotics and Remote Systems, Augusta, GA, April 1997.
Rastogi, A., "Design of an interface for teleoperation in unstructured environments using Augmented Reality displays", Unpublished MASc dissertation, University of Toronto, 1995.
Ruffo K., Milgram P., 'Effects of stereographic + stereovideo 'tether' enhancement for a peg in hole task', Proc. IEEE International Conference on Systems Man and Cybernetics, p.1425- 1420, Chicago, Oct. 1992.
Surdick R.T., Davis T.D., Kin R.A., Corso G.M., 'Relevant cues for the visual perception of depth: Is where you see it where it is?' Proc. Human Factors and Ergonomics Soc. 38th Annual Meeting, p.1305-1309, 1994.